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cient  condition  for  BGP  robustness  developed  by  Griffin  and  Wilfong.  In  this  thesis, 
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then  discuss  how  new  guidelines  for  configuring  BGP  with  a  guarantee  of  robustness 
may  be  derived  from  this  new  condition.  Additionally,  we  compare  various  models  of 
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I. 


INTRODUCTION 


Border  Gateway  Protocol  (BGP)  is  currently  the  only  interdomain  routing 
protocol  employed  on  the  internet.  It  allows  hundreds  of  thousands  of  autonomous 
systems  (ASes)  to  interconnect  by  providing  a  common  protocol  to  share  network 
reachability  information.  Within  an  autonomous  system,  shortest  path  routing  pro¬ 
tocols  are  sensible.  They  provide  a  predictable  method  of  routing  network  traffic  and 
usually  provide  optimal  routing.  However,  using  shortest  path  heuristics  to  route 
traffic  between  autonomous  systems  is  unattractive.  Each  AS  is  administered  by  an 
organizational  entity  that  may  have  a  range  of  economic  and  organizational  incen¬ 
tives.  Because  the  goal  of  many  ASes  is  to  earn  income  by  providing  internet  service, 
these  incentives  vary  widely  between  ASes.  Furthermore,  even  an  AS  which  does  not 
seek  to  gain  financially,  may  wish  to  limit  unnecessary  network  traffic  flow  so  that  it 
maintains  an  acceptable  level  of  service  to  its  users.  The  incentives  of  ASes  can  be 
expressed  in  terms  of  network  policies.  The  varying  policies  between  ASes  create  the 
need  for  a  protocol  which  does  not  rely  on  shortest  path  routing. 

BGP  has  been  widely  successful  because  it  gives  network  administrators  the 
ability  to  interconnect  with  other  ASes  and  implement  their  organization’s  policies. 
Unfortunately,  the  ability  of  BGP  to  implement  organizational  policies  may  also  lead 
to  routing  oscillations  and  unpredictable  routing  solutions  [Ref.  24]  when  ASes  have 
conflicting  policies.  We  describe  a  system  of  routers  as  robust  if  routing  tables  always 
converge  predictably,  under  any  set  of  router  and  link  failures. 

A.  THE  IMPORTANCE  OF  BGP  ROBUSTNESS 

Robustness  is  crucial  for  the  performance  of  the  internet  infrastructure.  Persis¬ 
tent  routing  oscillations  may  significantly  impact  end-to-end  performance,  resulting 
in  increased  latency  and  dropped  packets.  Persistent  routing  oscillations  also  make 
it  difficult  for  network  operators  to  identify,  debug,  and  correct  undesirable  rout- 
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ing  instances.  Furthermore,  robustness  is  crucial  for  maintaining  predictable  routing 
behavior.  If  routing  behavior  is  unpredictable,  optimal  routing  may  not  be  achieved. 


B.  SUMMARY  OF  THIS  PAPER 

A  number  of  approaches  have  been  pursued  to  address  BGP  instability.  This 
paper  investigates  achieving  robustness  of  eBGP  sessions  by  implementing  local  and 
global  constraints.  Using  the  stable  paths  problem  as  a  framework  for  BGP  polices 
[Ref.  12],  we  investigate  and  compare  various  BGP  models  to  show  that  they  do  not 
always  match  each  other.  We  present  new  sufficient  condition  for  robustness,  that 
is  weaker  than  any  previously  published  condition.  We  pursue  devising  constraints 
which  guarantee  this  condition.  We  also  apply  our  results  using  the  class-based  path- 
vector  system  [Ref.  18]. 

C.  ORGANIZATION  OF  THIS  PAPER 

The  remainder  of  this  paper  is  organized  as  follows. 

Chapter  II  gives  a  tutorial  of  BGP.  We  introduce  BGP  and  the  services  that 
it  provides.  We  describe  how  routers  establish  BGP  sessions  and  describe  the  various 
messages  that  can  be  exchanged.  We  discuss  how  BGP  allows  operators  to  implement 
network  policy.  We  discuss  how  routers  use  BGP  to  store,  select,  and  advertise  routes. 
We  define  three  major  design  goals  of  BGP:  autonomy,  expressiveness,  and  robustness. 
We  detail  how  permanent  routing  oscillations  may  arise  from  conflicting  policies.  We 
discuss  route  flap  dampening  as  the  current  solution  to  address  BGP  oscillations. 

Chapter  III  presents  background  work  that  addresses  achieving  BGP  robust¬ 
ness.  We  review  the  main  approaches  to  making  BGP  robust.  We  discuss  why  we 
pursue  an  approach  to  achieving  BGP  robustness  that  relies  on  operational  guidelines 
and  global  constraints.  We  give  a  summary  of  related  work  on  BGP.  We  reintroduce 
the  stable  paths  problem  as  a  framework  to  model  policies  and  routing  solutions  of 
BGP  systems.  We  define  solvability  as  the  existence  of  a  stable  routing  assignment. 
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We  describe  three  models  of  BGP  behavior:  the  simple  path  vector  protocol,  the 
single  node  activation  sequence  model,  and  the  multiple  node  activation  sequence 
model.  For  each  model  we  define  safety.  We  reintroduce  the  dispute  wheel  as  a 
sufficient  condition  for  the  robustness  of  the  stable  paths  problem.  A  dispute  wheel 
represents  a  set  of  mutually  conflicting  policies.  We  discuss  the  hierarchical  BGP 
model  which  describes  local  and  global  constraints  on  ASes  to  guarantee  robustness. 
We  reintroduce  the  class-based  path-vector  system  which  describes  generalized  local 
and  global  constraints  on  ASes  that  guarantee  robustness. 

Chapter  IV  compares  the  three  models  of  BGP  behavior.  We  describe  how  the 
models  match  each  other  in  terms  of  achieving  similar  successive  path  assignments 
when  given  the  same  instance  of  the  stable  paths  problem  and  initial  routing  tables. 
We  prove  that  while  some  models  match  each  other,  others  do  not.  We  compare  the 
definitions  of  safety  between  the  different  models.  We  prove  that  while  the  definition 
of  safety  in  one  model  may  imply  safety  in  another,  this  is  not  true  for  all  models. 

Chapter  V  gives  our  main  result  .  We  motivate  our  result  by  an  instance  of  the 
stable  paths  problem  which  is  robust,  but  contains  a  dispute  wheel.  We  introduce  a 
new  condition  on  instances  of  the  stable  paths  problem.  We  prove  that  this  condition 
is  robust,  and  weaker  than  previously  published  conditions.  We  investigate  applying 
our  result  using  the  class-based  path-vector  system  framework.  We  pursue  devising 
broader  guidelines  to  guarantee  robustness,  despite  the  presence  of  a  dispute  wheel. 

Chapter  VI  gives  conclusions  and  future  work. 
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II. 


TUTORIAL  OF  BGP 


Border  Gateway  Protocol  (BGP)  is  currently  the  only  interdomain  routing 
protocol  employed  on  the  internet.  The  internet  connects  tens  of  thousands  of  au¬ 
tonomous  systems  (ASes).  An  AS  is  a  collection  of  routers  controlled  by  a  single 
entity,  such  as  a  local  ISP  or  university.  It  is  also  common  for  very  large  organiza¬ 
tions  to  operate  more  than  one  AS.  Each  AS  is  given  a  globally  unique  number  called 
the  autonomous  system  number  (ASN).  Inside  an  AS  an  interior  gateway  protocol 
(IGP)  such  as  RIP  and  OSPF  is  used  to  determine  routes.  However,  ASes  communi¬ 
cate  with  each  other  using  BGP,  making  BGP  an  interdomain  protocol.  Specifically, 
BGP  gives  each  AS  the  ability  to  (1)  obtain  reachability  information  from  neighboring 
ASes  (2)  propagate  routing  information  and  (3)  choose  routes  based  on  reachability 
and  policy  [Ref.  22],  Unlike  OSPF  or  RIP,  routes  in  BGP  are  not  usually  determined 
by  shortest  path  metrics.  ASes  often  have  various  economic  incentives.  Because  BGP 
gives  network  administrators  an  enormous  amount  of  control  over  how  routes  are  ad¬ 
vertised  to  neighboring  ASes  and  how  routes  are  chosen,  BGP  is  often  referred  to  as 
“policy-based”  routing. 


Figure  1.  An  Small  Scale  Example  of  Internet  Routing 
To  begin  a  BGP  session,  a  BGP  speaker  establishes  a  TCP  connection  on 
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port  179  with  another  BGP  speaker  and  sends  an  OPEN  message.  There  are  two 
types  of  BGP  sessions.  An  interior  border  gateway  protocol  (iBGP)  session  allows 
an  autonomous  system  to  propagate  routing  information  within  itself.  An  exterior 
border  gateway  protocol  (eBGP)  session  allows  an  autonomous  system  to  share  rout¬ 
ing  information  with  a  different  AS,  also  known  as  an  external  peer.  See  Figure  1 
for  a  small  scale  example  of  routing  protocols  used  on  the  internet.  For  the  purposes 
of  this  paper,  we  will  ignore  the  complexities  of  iBGP  and  assume  that  each  AS  has 
completely  uniform  routing  information  at  any  given  time.  Therefore,  we  consider 
each  AS  as  a  single  entity  or  node  that  has  eBGP  sessions  with  external  peers. 

Once  a  BGP  session  has  been  established,  several  types  of  messages  are  sent 
between  BGP  speakers.  KEEPALIVE  messages  are  periodically  to  ensure  that  the 
connection  is  alive.  NOTIFICATION  messages  are  sent  in  response  to  errors  or 
special  conditions.  LIPDATE  messages  are  used  to  advertise  routes  between  BGP 
speakers.  A  route  is  a  set  of  destinations  with  information  about  the  path  to  those 
destinations.  UPDATE  messages  send  information  about  routes  by  using  the  Network 
Layer  Reachability  Information  (NLRI)  held  and  the  path  attributes  held[Ref.  22], 

The  path  attributes  held  of  an  UPDATE  message  allows  BGP  speakers  to 
share  detailed  information  about  routes.  We  will  briefly  discuss  some  of  the  most 
important  attribute  types,  including  AS_PATH,  ORIGIN,  MULTI_EXIT_DISC,  and 
LOCAL_PREF.  The  mandatory  AS_PATH  attribute  informs  the  local  BGP  speaker 
of  which  ASes  carried  the  routing  information  to  the  local  speaker.  If  this  routing 
information  has  not  changed,  these  same  ASes  will  carry  any  traffic  sent  to  the  route’s 
destination.  The  ability  to  share  the  AS_PATH  parameter  makes  BGP  a  path-vector 
protocol.  When  an  AS  shares  reachability  information  about  a  destination  to  one  of 
its  neighbors,  it  shares  the  entire  path  of  ASes  to  the  destination.  This  helps  prevent 
routing  loops,  because  no  path  will  ever  be  accepted  if  it  crosses  through  the  same 
AS  number  twice.  The  mandatory  ORIGIN  attribute  identifies  whether  the  original 
source  of  routing  information  was  from  an  interior  gateway  protocol,  the  exterior 
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gateway  protocol,  or  unknown.  The  optional  MULTI_EXIT_DISC  (MED)  attribute  is 
passed  between  external  peers  and  allows  a  local  AS  to  discriminate  between  multiple 
entry  and  exit  points  to  the  same  neighboring  AS.  The  LOCAL_PREF  attribute  must 
be  included  in  any  UPDATE  message  between  internal  peers.  This  attribute  helps 
an  AS  rank  paths  and  maintain  consistent  rankings  throughout  the  AS. 

As  discussed  above,  BGP  is  policy-based  routing.  BGP  operators  use  rankings 
and  filters  to  implement  their  policies.  A  BGP  speaker  may  have  a  multiple  routes 
to  a  single  destination  available.  Rankings  determine  which  of  these  routes  should 
be  used.  Also,  an  AS  may  not  want  to  share  all  of  its  routes  with  an  external  peer. 
Export  filters  allow  an  AS  to  place  controls  on  the  routes  advertised  to  external  peers. 
Conversely,  an  AS  may  not  want  to  use  some  of  the  routes  that  it  has  received.  Import 
filters  allow  an  AS  to  not  use  specified  routes. 

Rankings  are  determined  from  a  large  number  or  factors.  Phase  1  of  the 
decision  process  is  decision  function  that  is  invoked  whenever  a  BGP  speaker  recieves 
an  UPDATE  message,  from  a  peer,  that  advertises  a  new  route,  a  relacement  route,  or 
a  withdrawn  route.  Phase  1  calculates  the  degree  of  preference  for  each  newly  recieved 
or  replaced  route.  If  the  route  is  learned  via  an  iBGP  session,  either  the  LOCAL_REF 
attribute  is  is  taken  as  the  degree  of  preference  or  the  degree  of  preference  is  computed 
based  on  preconfigured  policy  information.  If  the  route  is  learned  via  an  eBGP  session, 
then  the  degree  of  preference  is  based  on  preconfigured  policy  information.  [Ref.  22], 

Phase  2  of  the  decision  process  is  invoked  immediately  after  Phase  1  and 
determines  which  routes  should  be  used  by  a  BGP  speaker.  AS  loops  are  detected 
by  scanning  the  full  AS  path  of  each  route  and  making  sure  that  none  of  these  ASNs 
matches  that  of  the  local  system.  [Ref.  22],  Also,  if  a  route  becomes  inaccessible,  it 
can  not  be  used.  Once  these  routes  have  been  eliminated,  the  highest  ranked  route 
is  selected  by  the  following  rules  in  their  exact  order  [Ref.  22]: 

1.  Prefer  the  path  with  the  largest  local  preference. 
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2.  Remove  from  consideration  all  paths  that  are  not  tied  for  having  the  largest 
local  preference. 

3.  Prefer  the  path  that  passes  through  the  smallest  number  of  ASes. 

4.  Remove  from  consideration  all  paths  that  are  not  tied  for  passing  through  the 
smallest  number  of  ASes. 

5.  Prefer  the  path  that  has  the  lowest  Origin  number. 

6.  Remove  from  consideration  all  paths  that  are  not  tied  for  having  the  lowest 
Origin  number. 

7.  Prefer  the  path  with  the  lowest  MED  attribute. 

8.  Remove  from  consideration  all  paths  that  are  not  tied  for  having  the  lowest 
MED  attribute. 

9.  If  at  least  one  path  was  received  via  EGP,  remove  from  consideration  all  paths 
that  were  received  via  IGP. 

10.  Prefer  the  route  with  the  most  preferred  interior  cost. 

11.  Remove  from  consideration  all  paths  that  are  not  tied  from  having  the  the 
most  preferred  interior  cost. 

12.  Prefer  the  route  with  the  lowest  BGP  identifier  value. 

13.  Prefer  the  path  with  the  lowest  external  peer  IP  address. 

Within  an  AS,  BGP  speakers  may  assign  routes  a  specific  local  preference 
value,  based  on  criteria  such  as  AS_PATH.  Because  local  preference  is  the  first  at¬ 
tribute  inspected  in  the  decision  process,  this  ability  allows  every  AS  to  rank  all  routes 
in  any  arbitrary  order. 

A  BGP  speaker  may  be  configured  to  filter  routes  in  a  number  of  ways.  Filters 
may  be  specified  by  ASNs  occurring  in  a  route’s  the  AS_PATH  attribute  and/or  the 
route’s  destination  address.  Filters  may  be  applied  to  prevent  a  route  from  entering 
the  router’s  routing  information  base.  This  would  prevent  a  specified  route  from  ever 
being  selected.  Filters  may  also  be  applied  to  prevent  a  route  from  being  sent  in  an 
UPDATE  message  to  an  external  peer. 


Now  that  we  have  discussed  ranking  and  filtering,  we  discuss  a  conceptual 
model  of  how  BGP  stores,  selects,  and  advertises  routes.  There  are  three  conceptually 
distinct  storage  tables  for  routes.  The  Adj-RIBs-In  table  contains  all  unprocessed 
routes  that  have  been  received  from  peers.  The  Loc-RIB  table  contains  each  actual 
route  used  locally  for  all  available  destinations.  This  is  determined  by  applying  import 
filters  and  rankings.  The  Adj-RIBs-Out  contains  the  routing  information  that  will 
be  shared  with  neighbors  in  outgoing  UPDATE  messages.  Suppose  a  BGP  speaker 
receives  a  route  from  a  peer  in  an  UPDATE  message.  The  BGP  speaker  will  store 
the  route  in  the  Adj-RIBs-In  table.  Next,  the  BGP  speaker  will  undergo  its  decision 
process  to  determine  if  this  received  route  should  be  used.  Routes  which  should 
be  filtered  and  routes  which  have  a  repeated  ASN  are  eliminated  from  the  decision 
process.  The  router  will  use  its  ranking  rules  to  determine  whether  the  received  route 
is  now  the  highest  ranked  route  to  a  destination.  If  this  is  the  case,  the  received  will 
replace  the  existing  route  in  the  Loc-RIB  table  and  the  BGP  speaker  will  begin  routing 
traffic  towards  the  first  hop  of  the  new  route.  Finally,  export  Liters  are  applied  to 
determine  whether  the  route  should  be  advertised  to  neighbors.  If  the  route  is  eligible 
to  be  advertised  to  neighbors,  the  route  will  be  updated  with  new  attributes  such  as 
AS_PATH  and  NEXT_HOP.  The  updated  route  and  eligible  neighbors  will  be  stored 
in  the  Adj-RIBS-Out  table.  UPDATE  messages  will  be  sent  containing  the  updated 
route. 

Routes  may  also  be  withdrawn  in  three  different  ways.  If  a  route  is  withdrawn, 
the  route  must  be  deleted  from  Adj-RIBS-In,  Loc-RIB,  and  Adj-RIBS-Out.  If  a  BGP 
speaker  deletes  any  route  from  the  Adj-RIBS-Out  table,  it  must  inform  its  neighbors 
that  this  route  is  no  longer  available.  A  route  can  be  withdrawn  by  sending  an 
UPDATE  message  with  the  route  placed  in  the  WITHDRAWN  ROUTES  field.  A 
route  can  be  withdrawn  by  advertising  a  new  route  that  contains  the  same  NLRI.  A 
route  can  be  withdrawn  by  closing  the  BGP  connection. 

The  ability  of  BGP  to  function  as  a  policy-based  protocol  leads  us  to  introduce 
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two  major  design  goals  of  BGP,  autonomy  and  expressiveness.  Autonomy  is  the 
ability  of  network  operators  to  make  policy  decisions  without  coordinating  with  other 
ASes.  Without  a  large  amount  of  autonomy,  network  operators  may  have  to  update 
their  policies  when  the  BGP  configurations  of  neighboring  ASes  change.  Furthermore, 
without  a  large  amount  of  autonomy,  network  administrators  of  different  ASes  may  be 
forced  into  a  situation  where  they  must  disclose  some  of  their  policies  to  each  other. 
Due  to  economic  incentives,  network  operators  often  require  that  they  keep  their  BGP 
policies  private.  Expressiveness  is  the  ability  of  network  operators  to  specify  network 
policy  in  a  flexible  manner.  For  instance,  shortest-path  routing  does  not  provide 
enough  expressiveness,  because  it  can’t  capture  the  economic  relationships  between 
many  ASes  such  as  customer,  provider,  and  peer  [Ref.  5]  [Ref.  If], 

In  general,  BGP  operators  configure  policies  in  line  with  their  organization’s 
economic  incentives,  which  are  determined  by  agreements  with  neighboring  ASes. 
Many  agreements  between  ASes  can  be  characterized  as  either  a  peer-to-peer  rela¬ 
tionship  or  a  customer-provider  relationship  [Ref.  17].  In  a  peer-to-peer  relationship, 
two  neighboring  ASes  benefit  from  exchanging  traffic  between  each  other’s  customers. 
When  BGP  relationships  are  discussed,  the  word  “peer”  will  refer  to  an  AS  which 
is  following  a  peer-to-peer  agreement  with  a  neighboring  AS.  In  a  customer-provider 
relationship,  one  neighbor  takes  on  the  role  of  customer  and  the  other  takes  on  the 
role  of  provider.  The  customer  pays  the  provider  for  access  to  internet  destinations 
that  could  not  be  otherwise  obtained  [Ref.  8].  If  an  organization  has  such  agreements, 
network  operators  may  implement  an  economically  advantageous  policy  by  adhering 
to  the  following  rules: 

1.  An  AS  can  advertise  only  the  routes  of  itself  and  its  customers  to  a  provider 
or  peer. 

2.  An  AS  can  advertise  all  known  routes  to  its  customers. 

The  first  rule  prevents  an  AS  from  carrying  traffic  without  receiving  compen¬ 
sation  or  benefit.  The  second  rule  allows  a  provider  to  inform  its  customers  of  routes 
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so  that  it  may  receive  compensation  for  carrying  traffic. 

Routing  Oscillations  occur  when  routers  exchange  streams  of  routing  updates 
that  do  not  reflect  any  change  to  network  topology  or  configuration.  Some  oscillations, 
such  as  the  RIP  vl  count  to  infinity  problem,  eventually  end  after  a  large  amount 
of  unnecessary  information  has  been  exchanged.  An  oscillation  that  will  eventually 
end  is  known  as  a  transient  oscillation.  Permanent  oscillations  occur  when  routers 
exchange  endless  streams  of  routing  updates,  and  may  be  created  by  conflicting  BGP 
policies  or  iBGP  configurations.  Routing  oscillations  may  use  up  router  processing 
power,  increase  network  latency,  cause  forwarding  loops  and  partition  the  network 
[Ref.  25].  Furthermore,  oscillations  can  be  exacerbated  by  failed  links  as  well  as 
complicate  the  diagnosis  and  debugging  of  network  problems  [Ref.  23].  Finally, 
routing  oscillations  may  significantly  affect  the  increasing  number  of  streaming  media 
applications  on  the  internet  today. 

Some  BGP  oscillations  may  arise  from  iBGP  configurations  alone.  Clustering- 
induced  divergence  occurs  when  an  interaction  between  route  reflection  clustering  and 
intradomain  routing  costs  causes  permanent  oscillations  [Ref.  15].  This  anomaly  may 
occur  even  when  eBGP  configuration  is  robust.  Griffin  et  al  [Ref.  15]  gave  a  sufficient 
condition  to  solve  this  problem,  which  is  based  upon  restricting  the  choices  of  paths 
at  some  routers.  In  another  type  of  iBGP  anomaly,  MED-induced  divergence  occurs 
when  an  interaction  between  MED  values,  route  reflection  clustering,  and  intradomain 
routing  costs  causes  permanent  oscillations  [Ref.  2],  Musunuri  and  Cobb  proposed 
routing  protocols  that  would  eliminate  this  anomaly  [Ref.  20]  . 

Besides  oscillations  occurring  from  iBGP,  eBGP  may  also  cause  oscillations. 
When  multiple  BGP  speakers  have  conflicting  routing  policies,  there  may  be  perma¬ 
nent  oscillations.  To  see  how  router  configuration  may  lead  to  permanent  routing 
oscillations,  consider  a  case  where  there  are  four  eBGP  speakers  named  “0” ,  “1” ,  “2” , 
and  “3”  with  the  unique  ASNs  100,  101,  102,  and  103  respectively.  The  system  has 
the  configuration  as  depicted  in  Figure  2.  We  are  interested  in  routing  a  particular 
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packet  to  a  destination  d  inside  ASN  0.  Therefore,  we  are  interested  in  routes  that 
have  an  AS  Path  that  ends  in  100.  With  that  in  mind,  each  router  is  configured  to 
have  the  routes  and  preferences  as  depicted  in  Figure  3. 


Figure  2.  Configuration 


“I  know  about  two 
routes  to  router  0.  I 
know  I  can  go 
through  router  2  with 
route  (1  2  0)  or 
directly  with  route 
(1  0).  I  prefer  going 
through  router  2.” 


(12  0) 
(2  0) 


Figure  3.  Available  Routes 

The  router  configurations  can  be  described  as  follows.  Router  0  exports  all 
routes  to  routers  1,  2,  and  3.  Router  1  exports  all  routes  to  routers  0,  1,  and  2. 
Router  1  filters  all  routes  received  from  router  3  and  filters  the  single  route  received 
from  router  2  that  has  the  AS  Path  101  102  103  100.  Router  1  prefers  the  route  with 
the  AS  Path  101  102  100  to  the  route  with  the  AS  Path  101  100.  Router  2  exports 
all  routes  to  routers  0,  1,  and  3.  Router  2  filters  all  routes  received  from  router  1  and 
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filters  the  single  route  received  from  router  3  that  has  the  AS  Path  102  103  101  100. 
Router  2  prefers  the  route  with  AS  Path  102  103  100  to  the  route  with  AS  Path  102 
100.  Router  3  exports  all  routes  to  routers  0,  1,  and  2.  Router  3  filters  all  routes 
received  from  router  2  and  filters  the  single  route  received  from  router  1  with  AS 
Path  103  101  102  100.  Router  3  prefers  the  route  with  AS  Path  103  101  100  to  the 
route  with  AS  path  103  100.  For  an  example  configuration  hie  for  router  1,  see  the 
appendix. 


Figure  4.  The  Steps  in  the  Permanent  Oscillation 


The  router  configuration  discussed  above  will  always  give  rise  to  permanent 
routing  oscillations.  We  will  go  through  a  sequence  of  routing  updates  to  show  the 
permanent  oscillations  as  depicted  in  Figure  4. 

1.  Router  1  routes  through  router  2  to  router  0.  Router  2  routes  directly  to  router 
0.  Router  3  routes  directly  to  router  0.  Also,  an  UPDATE  message  has  been 
sent  from  router  3  to  router  2,  informing  router  2  of  its  new  route  to  router  0. 
However,  router  2  has  not  received  this  message  yet. 

2.  Router  2  receives  and  processes  the  UPDATE  message  from  router  3.  Router 
2  changes  its  route  to  route  through  router  3  to  router  0.  Router  2  sends  an 
UPDATE  message  to  router  1  informing  router  1  of  its  new  route.  However, 
router  1  has  not  processed  this  message  yet. 
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3.  Router  1  receives  and  processes  the  UPDATE  message  from  router  2.  Router 
l’s  current  route  to  router  0  through  router  2  is  no  longer  available  and  the 
new  route  received  from  router  2  is  filtered.  Therefore  router  1  changes  its 
route  and  routes  directly  to  router  0.  Router  1  sends  an  UPDATE  message  to 
router  3  informing  router  3  of  its  new  route. 

4.  Router  3  receives  and  processes  the  UPDATE  message  from  router  f.  Router 
3  changes  its  route  and  routes  through  router  1.  Router  3  sends  and  UPDATE 
message  to  router  2  informing  router  2  of  its  new  route. 

5.  Router  2  receives  and  processes  the  UPDATE  message  from  router  3.  Router 
2’s  current  route  to  router  0  is  no  longer  available.  Router  2  changes  its  route 
to  route  directly  to  router  0.  Router  2  sends  an  UPDATE  message  to  router 

1  informing  router  1  of  its  new  route. 

6.  Router  1  receives  and  processes  the  update  message  from  router  2.  Router  f 
changes  its  route  and  routes  through  router  2.  Router  f  sends  an  UPDATE 
message  to  router  3  informing  router  3  of  its  new  route. 

7.  Router  3  receives  and  processes  the  update  message  from  router  1.  Router  3’s 
current  route  is  no  longer  available.  Router  3  changes  its  route  to  route  directly 
to  router  0.  Router  3  sends  an  UPDATE  message  to  router  2  informing  router 

2  of  its  new  route.  Note  that  at  the  end  of  step  7  we  are  in  the  exact  same 
state  as  in  the  end  of  step  1. 

Because  Step  f  and  Step  7  result  in  the  exact  same  state,  this  process  will 
repeat  itself  indefinitely.  If  a  system  of  routers  is  always  guaranteed  to  converge  and 
stop  changing  routes,  no  matter  what  order  messages  are  processed  in,  the  system  is 
known  as  safe.  The  example  we  have  just  examined  is  not  safe.  Solvability  is  another 
characteristic  of  systems  of  BGP  routers  that  does  not  always  hold.  A  system  of 
routers  is  solvable  if  there  exists  a  set  of  system  wide  routing  tables  where  if  any 
router  receives  a  correct  UPDATE  message,  that  router  will  not  change  its  current 
routing  table.  The  example  we  have  just  examined  is  not  solvable.  Unique  solvabil¬ 
ity  is  a  more  stringent  characteristic  where  there  is  exactly  one  set  of  system-wide 
routing  tables  that  are  solvable.  If  a  system  of  routers  is  uniquely  solvable  and  safe, 
the  system  is  guaranteed  to  converge  in  a  predictable  manner. 

Now  that  BGP  oscillations  have  been  discussed,  we  introduce  robustness  as 
the  third  major  design  goal  of  BGP.  Robustness  is  a  characteristic  where  router  con- 
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figuration  can  not  lead  to  routing  oscillations  and  must  always  produce  a  predictable, 
unique  routing  solution,  under  any  set  of  link  and  router  failures.  For  BGP  to  be 
robust,  constraints  must  be  put  on  the  expressiveness  and  autonomy  of  the  protocol 
[Ref.  11]. 

In  order  to  minimize  the  effects  of  BGP  oscillations,  route  flap  dampening 
[Ref.  25]  is  often  employed.  Route  flap  dampening  is  an  extension  to  BGP  that  allows 
routers  to  maintain  information  on  the  stability  of  individual  routes.  A  BGP  speaker 
will  suppress  routes  that  show  a  large  degree  of  instability.  Also,  fixed  timers  may 
be  used  to  slow  route  advertisement.  While  route  flap  dampening  may  successfully 
minimize  some  of  the  adverse  effects  of  oscillations,  it  does  not  provide  a  complete 
solution.  Route  flap  dampening  causes  oscillations  to  run  in  slow  motion  and  does 
not  guarantee  that  routing  tables  will  converge  to  a  predictable,  unique  solution. 

In  this  chapter,  we  have  examined  how  BGP  uses  rankings  and  filterings  to 
select  routes  and  implement  network  policy.  We  have  described  how  conflicts  in 
policies  may  create  BGP  oscillations. 
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III.  BACKGROUND  WORK 

A.  APPROACHES  TO  MAKING  EBGP  ROBUST 

There  are  currently  three  main  approaches  to  address  the  instability  of  eBGP 
[Ref.  12].  The  approaches  consist  of  operational  guidelines  for  BGP  operators,  static 
analysis  of  routing  policies,  and  modification  of  the  BGP  protocol. 

If  every  BGP  operator  followed  the  same  set  of  operational  guidelines,  it  is 
possible  to  prove  the  robustness  of  BGP  for  certain  sets  of  guidelines.  For  instance, 
if  every  BGP  operator  configured  policies  using  route  filtering  alone,  BGP  is  guaran¬ 
teed  to  be  a  robust  protocol  [Ref.  12].  Another  flexible,  but  complex  set  of  robust 
guidelines  are  proposed  by  Gao  and  Rexford  [Ref.  8]. 

There  are  a  number  of  downsides  to  relying  on  operational  guidelines.  First, 
not  all  BGP  operators  may  follow  such  operational  guidelines.  A  set  of  operational 
guidelines  may  not  capture  every  policy  that  BGP  operators  may  be  interested  in  im¬ 
plementing  or  BGP  operators  may  ignore  operational  guidelines  altogether.  Second, 
the  set  of  robust  operational  guidelines  may  be  overly  strict.  There  may  exist  config¬ 
urations  of  routers  that  are  robust,  despite  the  fact  they  do  not  implement  any  known 
operational  guidelines.  Third,  operational  guidelines  may  require  BGP  operators  to 
disclose  some  amount  of  policy  with  each  other  in  order  to  check  global  constraints. 
For  reasons  already  noted,  most  BGP  operators  are  very  reluctant  to  disclose  their 
configurations  with  each  other. 

In  another  approach,  BGP  robustness  could  be  achieved  by  static  analysis  of 
router  configurations.  Such  a  solution  would  analyze  the  configuration  of  all  BGP 
speakers  and  look  for  policy  conflicts.  This  solution  has  been  proposed  by  Govindan 
et  al.  [Ref.  10].  There  are  at  least  two  major  drawbacks  to  this  approach.  First, 
BGP  operators  would  have  to  disclose  the  policies  and  configurations  of  their  AS  with 
each  other.  For  economical  reasons,  most  BGP  operators  are  very  reluctant  to  disclose 
their  configurations.  Second,  such  an  approach  is  likely  to  be  intractable,  without  any 
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heuristic  procedure  to  check  convergence  properties  or  constraints  on  ASes.  Griffin 
and  Wilfong  have  shown  that  checking  for  global  convergence  conditions  is  either 
NP-complete  or  NP-hard  [Ref.  16]. 

In  the  final  approach,  the  BGP  protocol  could  be  modified  to  suppress  or 
prevent  eBGP  oscillations.  This  approach  is  sometimes  referred  to  as  a  dynamic 
approach,  because  it  happens  at  run-time.  As  discussed  in  Chapter  II,  the  route-flap 
dampening  [Ref.  25]  can  suppress  eBGP  oscillations.  Unfortunately,  this  approach 
only  makes  oscillations  run  in  slow  motion  and  does  not  guarantee  that  BGP  will 
converge  to  a  predictable,  unique  solution. 

More  extensive  modifications  to  BGP  have  also  been  proposed.  Griffin  and 
Wilfong  propose  a  modification  to  BGP  where  an  attribute  called  path  history  is 
used  to  identify  paths  whose  histories  contain  cycles.  This  attribute  is  exchanged 
between  BGP  speakers.  Once  these  paths  have  been  identified,  the  modified  protocol 
can  also  suppress  such  paths  [Ref.  14],  Another  somewhat  similar  modification  to 
BGP  has  been  proposed  by  Tien  Ee  et  al.  [Ref.  4],  They  proposed  a  mechanism 
whereby  route  advertisements  are  tagged  by  a  global  precedence  value.  When  a  BGP 
speaker  advertises  this  route  to  its  neighbors,  it  will  increment  this  value  by  a  number 
corresponding  to  its  LOCAL_PREF  for  that  route.  If  permanent  BGP  oscillations 
occur,  routers  will  rely  on  these  global  precedence  values  instead  of  the  local  degree 
of  preference,  creating  a  stable  path  assignment. 

There  are  several  drawbacks  to  solutions  which  modify  BGP.  First,  in  the 
two  previous  BGP  modifications  discussed,  every  BGP  speaker  must  implement  the 
proposed  protocol  to  prevent  BGP  oscillations.  There  are  hundreds  of  thousands 
of  BGP  speakers  deployed  on  the  internet  today  and  operators  may  be  unwilling  to 
update  their  routers  to  new  standards.  Second,  protocol  modifications  that  suppress 
routes  dynamically  are  unpredictable  by  nature.  Often,  it  is  impossible  to  predict 
exactly  which  BGP  speaker  will  begin  suppressing  routes  related  to  permanent  oscil¬ 
lations,  eliminating  the  possibility  of  robustness.  Finally,  protocol  modifications  that 
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suppress  routes  involved  in  conflicting  policies  often  sacrifice  a  large  degree  of  trans¬ 
parency  [Ref.  1 1] .  Transparency  is  the  ability  of  BGP  operators  to  to  understand 
how  the  policies  they  have  written  affect  the  routing  protocol  and  routing  tables. 
When  dynamic  solutions  suppress  routes,  it  becomes  difficult  for  BGP  operators  to 
maintain  and  debug  routing  policies. 

Based  upon  the  above  discussion,  we  pursue  an  approach  that  relies  on  oper¬ 
ational  guidelines  along  with  global  constraints  in  order  to  achieve  robustness. 

B.  RELATED  WORK 

Bertsekas  [Ref.  1]  proved  that  the  distributed  Bellman-Ford  algorithm  con¬ 
verges.  Because  BGP  has  the  ability  to  employ  policy  based  routing,  this  proof  of 
shortest  path  routing  does  not  apply  to  BGP  in  general. 

Varadhan  et  al.  [Ref.  24]  first  observed  that  conflicting  policies  in  BGP  con¬ 
figuration  could  lead  to  persistent  routing  oscillations.  Furthermore,  they  introduced 
the  concept  of  safety,  by  defining  an  AS  as  “safe”  if  the  policy  of  an  AS  does  not  cause 
oscillations.  They  also  speculated  that  only  shortest  path  route  selection  is  provably 
safe. 

Labovitz  et  al.  [Ref.  19]  presented  results  from  a  two  year  long  study  of  inter¬ 
net  routing  convergence.  They  discussed  the  theoretical  upperbound  of  convergence 
time  for  certain  systems.  They  showed  that  when  routing  faults  were  injected  into 
the  internet,  convergence  took  much  longer  than  previously  thought. 

Feigenbaum,  Sami,  and  Shenker  [Ref.  6]  showed  that  systems  with  next  hop 
rankings  always  have  at  least  one  stable  routing.  However,  because  of  the  distributed 
nature  of  BGP,  such  systems  are  not  guaranteed  to  converge  to  a  stable  routing.  We 
give  an  example  such  a  system  in  Figure  10. 

Gao  and  Rexford  [Ref.  8]  introduced  sufficient  conditions  on  topology,  filter¬ 
ing,  and  rankings  to  guarantee  routing  stability  and  safety.  These  conditions  reflect 
the  real-world  configuration  of  autonomous  systems.  They  introduced  and  defined 
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the  activation  sequence  in  order  to  model  the  behavior  of  BGP.  They  developed  a 
system  of  constraints  based  upon  the  principle  that  every  autonomous  system  should 
regard  each  of  its  neighbors  as  either  a  provider,  a  customer,  or  a  peer.  Furthermore, 
they  defined  a  series  of  constraints  based  on  each  of  these  relationships.  Finally, 
they  proved  that  if  every  AS  follows  these  constraints,  stable  internet  routing  can  be 
achieved  without  global  coordination.  Unfortunately,  ASes  do  not  always  follow  such 
guidelines.  Further  work  by  Gao  [Ref.  7]  showed  that  some  small  ISPs  do  not  follow 
the  guidelines. 

Griffin,  Shepherd,  and  Wilfong  [Ref.  12]  introduced  the  dispute  wheel  as  a 
sufficient  condition  for  robustness.  They  defined  the  stable  paths  problem  (SPP), 
which  is  discussed  in  more  detail  in  this  chapter.  They  also  used  the  simple  path 
vector  protocol  (SPVP)  [Ref.  9]  to  model  the  behavior  of  BGP.  They  showed  that 
determining  the  solvability  of  SPP  is  an  NP-complete  problem.  Furthermore,  they 
introduced  the  dispute  wheel.  They  proved  that  the  absence  of  a  dispute  wheel  is  a 
sufficient  condition  for  SPP  solvability,  safety,  and  robustness. 

Griffin,  Jaggard,  and  Ramachandran  [Ref.  11]  introduced  a  framework  to 
describe  class-based  path-vector  systems.  They  detailed  a  method  where  matrices 
are  used  to  describe  the  scoping  (also  known  as  filtering)  and  ranking  rules  of  an 
AS  based  upon  its  relationships  with  neighboring  ASes  and  hierarchical  level.  They 
showed  how  the  framework  could  be  used  to  describe  conditions  on  relationships 
like  those  proposed  by  Gao  and  Rexford  [Ref.  8].  They  also  discussed  the  design 
goals  for  path-vector  protocols  like  BGP.  They  showed  that  in  order  to  guarantee 
robustness,  there  is  an  inherent  tradeoff  between  expressiveness  and  the  need  for 
global  constraints.  They  showed  that  if  full  autonomy  was  allowed  in  a  system, 
autonomous  systems  could  only  express  rankings  based  on  shortest  paths. 

Jaggard  and  Ramachandran  [Ref.  18]  continued  work  on  class-based  path- 
vector  systems  by  giving  specific  global  constraints  on  a  system  that  guarantee  ro¬ 
bustness.  They  proved  an  exact  global  condition  for  the  creation  of  a  dispute  wheel. 
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Furthermore,  they  gave  polynomial-time  central  and  distributed  algorithms  to  enforce 
this  constraint.  Unfortunately,  their  constraint  is  not  likely  to  be  the  most  general 
constraint  for  path- vector  systems. 

Feamster,  Johari,  and  Balakrishnan  [Ref.  5]  explored  the  inherent  tradeoff 
between  autonomy  and  expressiveness.  They  showed  that  next-hop  rankings  were 
not  safe. 

C.  THE  STABLE  PATHS  PROBLEM 

The  Stable  Paths  Problem  (SPP)  captures  the  apparent  routing  policies  over 
a  network  of  autonomous  systems  running  BGP  [Ref.  12]. 

The  SPP  framework  is  designed  to  describe  the  most  important  features  of 
path  selection  in  BGP.  The  SPP  framework  consists  of  a  simple,  bidirectional  graph 
G,  which  contains  a  collection  of  vertices  V  and  edges  E.  There  is  a  vertex  denoted  0 
which  represents  the  origin.  Every  other  vertex  is  interested  in  Ending  a  path  to  the 
origin.  For  each  vertex  v  G  V,  Vv  represents  a  set  of  paths  that  are  available  from 
that  vertex  v  to  the  origin  0. 

The  SPP  framework  also  includes  A  ,  which  is  a  ranking  function  on  the  paths 
Vv  available  at  each  vertex  v  G  V  —  {0}.  Let  V  be  the  set  of  all  paths  available  at 
all  vertices.  Because  the  set  of  routes  Vu  available  at  each  vertex  u  may  be  limited, 
SPP  captures  the  ability  of  each  AS  to  filter  routes.  However,  the  SPP  framework 
does  not  specify  whether  a  route  has  been  filtered  by  an  import  filter  or  an  export 
filter.  For  each  node  v,  there  is  a  ranking  function  Xv,  that  is  defined  over  Vv .  Let 
A  =  {At’|u  G  U  — {0}}.  For  each  such  node  v  ,  if  Pi,  P2  G  Vv  and  Xv(Pi)  >  XV(P2)  then 
node  u  is  said  to  prefer  the  path  Pi  over  the  path  P2.  The  ranking  function  A  captures 
the  ability  of  each  AS  to  autonomously  and  expressively  rank  routes.  Formally,  an 
instance  of  the  stable  paths  problem  denoted  S  is  expressed  as  a  triple  S  =  ( G ,  V 
,A). 
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Finally,  we  make  several  assumptions  about  the  paths  permitted  at  every  node 
and  the  ranking  function.  We  assume  that  V°  =  {(0)}.  For  all  u  G  W — {0}  we  assume: 

1.  If  a  path  is  permitted  P  G  Vu ,  then  P  is  a  simple  path,  (simplicity,  no  repeated 
nodes) 

2.  e  G  Vu  (empty  path  permitted) 

3.  Au(e)  =  0  and  VP  G  P"  such  that  P  ^  e,  A U(P)  >  0(empty  path  lowest 
ranked) 

4.  If  Pi  ,  P-2  G  P“,  Pi  ^  p2  ,  and  A“(Pi)  =  \U(P2),  then  G  V  such  that 
Pi  =  (uw)P\  and  P2  =  (mt^Pj  where  P(  and  P2  are  subpaths  of  Pi  and  P2 
respectively,  (strictness,  two  identically  ranked  paths  have  the  same  next  hop) 

Rule  1  captures  the  fact  that  BGP  eliminates  paths  with  repeated  AS  numbers. 
Rule  2  captures  the  fact  that  it  is  possible  for  every  AS  to  not  be  able  to  reach  any 
arbitrary  destination.  Rule  3  captures  the  fact  that  an  AS  will  take  any  allowed  and 
available  path  to  a  destination  rather  than  leave  the  destination  unreachable.  Rule  4 
captures  the  fact  that  when  an  AS  receives  routes  from  two  different  ASes,  one  route 
must  be  preferred  over  another. 

Figure  5  gives  a  pictorial  representation  of  the  SPP.  In  this  figure  we  see  that 
the  vertices  V  consist  of  {0, 1,  2,  3}  and  the  edges  E  consist  of  {(10)(12)(13) (20) (23) (30)} 
At  vertex  1,  the  paths  to  the  origin  (10)  and  (120)  are  available.  Vertex  1  would  prefer 
to  reach  the  origin  through  vertex  2  by  using  path  (120)  rather  than  reach  the  origin 
directly  using  path  (10). 

A  path  assignment  7r  is  a  function  that  maps  each  node  u  £  V  to  a  path 
7 t(u)  G  Vu .  The  set  of  paths,  choices^,  u),  is  defined  to  be 

choice^, u)  =  |  e«)nr  «  *  o 

Note  that,  only  the  path  of  length  1,  (0),  is  allowed  at  the  origin. 

Suppose  7 Zu  C  Vu  such  that  each  path  has  a  distinct  next-hop.  The  best  path 
in  VP  is  defined  to  be 
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Figure  5.  A  Pictorial  Representation  of  SPP 


best  (7 Zu) 


P  e  1ZU  with  maximal  \U(P)  1ZU  ^  0 
e  nu  =  0 


The  path  assignment  7r  is  stable  at  node  n  if  n(u)  =  best ( choices (7r,  u)).  The 
path  assignment  n  is  stable  if  it  is  stable  at  each  node  «6  0.  As  mentioned  in  [Ref. 
12]  any  stable  path  assignment  also  describes  a  tree  containing  the  origin. 

An  instance  of  SPP  is  solvable  if  there  exists  a  stable  path  assignment  for  the 
instance.  An  instance  of  SPP  is  uniquely  solvable  if  there  exists  exactly  one  stable 
path  assignment  of  the  instance. 

Deriving  subinstances  of  SPP  will  be  used  in  later  sections  represent  an  in¬ 
stance  of  the  stable  paths  problem  where  nodes  or  links  have  failed.  Given  an  instance 
of  SPP  S  =  ( G ,  V ,  A),  where  G  =  ( V. ,  E ),  there  is  a  natural  way  to  derive  subinstances 
of  SPP  from  subsets  of  E.  Suppose  E'  C  E,  we  define  SPP (E1)  =  (Ge'jVe'^e1)  to 
be  the  derived  instance  of  SPP  from  E' .  Let  the  graph  be  Ge1  =  (V,E')  .  Let  the 
set  of  available  paths  PE >  =  {P\P  G  V}  and  every  edge  in  the  path  P  is  present  in 
E’  .  For  each  node  u,  we  will  denote  its  set  of  available  paths  as  V Finally,  let  the 
ranking  function  be  A#'  =  A,  but  modified  to  exclude  all  omitted  paths. 
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D.  MODELS  OF  BGP  BEHAVIOR 

There  are  three  models  of  BGP  behavior,  the  simple  path  vector  protocol,  the 
single  node  activation  sequences  model,  and  the  multiple  node  activation  sequences 
model.  All  three  models  can  be  expressed  in  terms  of  the  stable  paths  problem 
and  can  express  how  BGP  speakers  exchange  UPDATE  messages  and  update  their 
routing  tables.  Furthermore,  all  models  have  specific  definitions  for  safety,  which  are 
all  conceptually  equivalent  to  BGP  safety.  In  Chapter  IV,  we  will  investigate  the 
equivalence  of  the  models. 

1.  Simple  Path  Vector  Protocol 

The  simple  path  vector  protocol  (SPVP)  captures  the  most  important  behav¬ 
ioral  characteristics  of  BGP  [Ref.  13]  [Ref.  16]  .  It  is  a  distributed  algorithm  which 
tries  to  solve  the  stable  paths  problem.  The  protocol  will  always  diverge  if  an  instance 
of  SPP  is  not  solvable.  However,  as  we  will  see  later,  the  protocol  can  also  diverge 
for  an  instance  of  the  stable  paths  problem  that  is  solvable. 

It  will  be  necessary  to  reintroduce  much  of  the  notation  from  [Ref.  12].  Each 
node  u  can  store  information  about  paths  in  two  different  data  structures.  The 
data  structure  rib(u)  stores  u' s  current  path  to  the  origin  or  7 t(u).  For  each  node 
u  and  a  stable  paths  problem  S,  we  define  the  set  of  nodes  peers(u)  to  be  the  set 
{v\(u  v )  G  E}.  For  each  w  G  peers(u),  the  data  structure  rib-in(u  4=  w)  stores 
the  most  recently  received  and  processed  path  from  w.  Because  we  do  not  assume 
messages  are  processed  immediately,  it  is  possible  that  rib- in (u  4=  w)  might  contain  a 
different,  older  path  than  rib(w).  Therefore,  we  define  the  choices  of  paths  available 
for  a  node  running  SPVP  slightly  differently  than  we  do  for  the  stable  paths  problem 
in  general.  Under  SPVP,  we  define  the  path  choices  available  at  node  u  to  be: 

SPVP-choices('u)  =  {(u  w)P  G  VU\P  =  rib-in(u  <4=  w)} 

Finally,  we  define  the  best  possible  path  that  is  available  to  u  as 
SPVP-best(u)  =  best.(SPVP-choices(«)) 
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process  spvp(-u) 
begin 

receive  P  from  w  — > 
begin 

rib-in(w  4=  w)  :=  P 

if  rib(u)  7^  SPVP-best.(u)  then 

begin 

rib(u)  :=  SPVP-best.(u) 
for  each  v  G  peers (u)  do 

begin 

send  rib(u)  to  r 

end 


end 


end 

end 


Figure  6.  SPVP  Process  at  Node  u  from  [Ref.  12]  ) 

This  path  is  the  highest  ranked  path  node  u  can  use  given  the  messages  that 
have  been  received  and  processed  from  its  peers. 

Figure  1  shows  how  SPVP  runs  for  each  node  u  G  V.  If  there  is  an  unprocessed 
message  from  any  w  G  peers(u),  the  guard  receive  P  from  w  can  be  activated  to 
receive  the  oldest  unprocessed  message  that  iv  has  sent  containing  path  P.  If  there 
are  multiple  links  with  unprocessed  messages,  any  link  may  be  selected.  When  the 
guard  is  activated  the  message  is  deleted  from  the  link  and  processed  in  one  atomic 
step  according  to  the  code  following  .  The  code  will  store  the  message  in  rib- 
in  (it  <7=  w).  If  the  current  selected  path  is  no  longer  the  best  available  path,  the  code 
will  change  the  current  selected  path  to  be  the  best  available  path  by  executing  rib(u) 
:=  SPVP-best.(u).  Finally,  it  will  send  this  path  to  all  neighbors,  v  G  peers(u). 

We  use  the  exact  notation  as  presented  in  [Ref.  12]  to  model  how  the  protocol 
operates  as  the  system  in  general.  Informally,  we  describe  the  network  state  of  the 
system  as  all  values  of  rib(u),  rib- in  (it  •<=  w),  and  the  state  of  all  communication 
links.  The  current  path  assignment  at  each  node  implicitly  defines  a  path  assignment 
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for  the  entire  system  if  7 t(u)  =  rib(u). 

We  model  (logical)  time  t  with  discrete  values  0,1,2,....  For  each  node  u 
and  each  w  G  peers(u),  mq(«  4=  w,t)  denotes  the  state  of  the  communication  link 
from  node  w  to  node  u  at  time  t.  This  is  a  FIFO  message  queue,  and  the  notation 
mq(«  <=  w,  t)  [i]  refers  to  the  ith  element  of  the  queue.  In  particular,  mq(«  -4=  w,  t)  [1] 
is  the  first  or  oldest  unprocessed  message  in  the  communication  link.  For  each  u, 
rib(u,t)  denotes  the  value  of  rib(u)  at  time  t.  For  each  u.  and  each  w  G  peers(u), 
rib-in(u  -4=  w,t)  denotes  the  value  of  rib-in (?i  4=  w)  at  time  t. 

The  network  state  at  time  t,  denoted  s(t),  is  comprised  of  all  values  rib(u,t), 
rib-in(w  4=  w,  t )  ,  and  mq(«  <=  w,t)  . 

At  each  state  transition  from  s(t  —  1)  to  s(t)  either  (1)  the  network  state 
remains  unchanged,  or  (2)  some  node  u  processes  a  message  from  some  w  G  peers(u). 
Note  that  at  each  transition,  only  one  node  processes  a  message  at  a  time.  We  define 
(j  as  a  sequence  of  nodes,  where  at  each  time  t.  a  the  tth  node  of  the  sequence  is 
activated  and  processes  one  message.  Let  s0  =  s(0)  be  some  initial  state  of  path 
assignments,  rib-in’s  and  message  queues.  We  describe  a  as  fair  with  respect  to  s0  if 
any  message  sent  from  a  node  w  to  a  node  u  will  eventually  be  processed. 

Definition:  Safe  (SPVP)  A  stable  paths  problem  is  called  safe  if  the  pro¬ 
tocol  SPVP  always  converges,  for  any  intial  state  s0  and  any  fair  sequence  a.  If  at 
time  t  the  network  state  s(t)  is  such  that  all  message  queues  mq(«  4=  v,t)  are  empty 
then  we  say  the  system  has  converged  at  time  t,  and  write  S(cr,  s0,t)  j. 

More  detail  about  SPVP  may  be  found  in  [Ref.  12]. 

2.  Single  Node  Activation  Sequences  Model 

Several  models  have  been  proposed  that  model  BGP  behavior  that  rely  on  one 
or  more  nodes  being  activated  at  a  given  point  of  time.  When  a  node  is  activated,  this 
abstractly  corresponds  to  the  node  receiving  instantaneous,  simultaneous  UPDATE 
messages  from  all  neighbors  and  selecting  the  best  available  path.  Feamster  et  al. 
proposed  a  BGP  model  (“Routing  protocol  dynamics”)  based  on  activating  only  a 
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single  node  at  a  time  [Ref.  5] .  Feamster  also  included  a  framework  to  describe  BGP 
filtering  and  ranking.  However,  the  Single  Node  Activation  Sequence  Model  (SNASM) 
will  be  described  in  terms  of  the  stable  paths  problem. 

Definition:  Infinitely  Often  For  a  sequence  of  elements  a  =  oq,  02,  <23...,  an 
element  b  is  said  to  appear  infinitely  often  if  the  element  b  is  repeated  in  the  sequence 
infinitely  many  times. 

Definition:  Fair  Single  Node  Activation  Sequence  A  sequence  of  nodes 
u)  =  Ui,U2,u3...  is  said  to  be  a  fair  single  node  activation  sequence  if  each  node  iq  G  U 
and  appears  infinitely  often  in  the  sequence. 

In  order  to  introduce  the  SNASM,  it  will  be  necessary  to  redefine  some  func¬ 
tions  in  order  to  introduce  the  concept  of  discrete  time.  We  define  the  path  assignment 
of  all  nodes  at  time  t  as  n(t)  (a  mapping  from  V  toV).  We  define  the  path  assignment 
at  a  particular  node  u  at  time  t  as  7T (u,t). 

The  set  of  available  paths  choices(n  (t),u,t)  from  node  u  at  a  particular  time  t 
is  defined  to  be 

choicesMO.u,*)  =  |  6  E]  n  u?  0 

Figure  7  presents  the  SNASM  Routing  Protocol  Dynamics.  Time  is  modeled 
discretely.  The  model  begins  with  an  initial  path  assignment  at  time  0  which  is 
7r(0).  The  model  uses  a  fair  single  node  activation  sequence  to  represent  the  fact  that 
in  BGP,  each  BGP  speaker  will  always  be  ready  to  receive  and  process  UPDATE 
messages  from  its  peers.  At  each  time  t  a  node  Ut  is  activated.  This  corresponds 
to  the  node  receiving  all  the  current  path  assignments  of  neighbors  simultaneously 
and  instantaneously.  The  node  will  then  pick  its  highest  ranked  and  available  path. 
Clearly,  the  model  defines  a  sequence  of  path  assignments  7r(0),  7t(1),  7t(2),  ...  for  each 
time  t  —  0, 1,  2, ...  This  model  differs  from  SPVP  because  it  does  not  take  into  account 
that  messages  may  be  in  transit,  and  may  be  processed  in  different  orders  if  they  are 
from  different  neighbors.  However,  this  model  can  take  into  account  the  fact  that 
when  a  node  changes  path,  the  path  it  changes  to  may  no  longer  actually  be  available. 
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For  instance,  suppose  a  node  v  is  being  activated  and  takes  the  highest  ranked  path 
from  a  node  w  of  the  form  {v  w  z)P.  It  is  possible  that  z  has  changed  its  path  since 
w  was  last  activated.  While  w  advertises  the  path  as  available,  it  isn’t. 


SNASM  Routing  Protocol  Dynamics 

At  time  t  —  1,  the  current  path  assignment  is  7r (t  —  1).  Each  node  u  has 
currently  selected  path  n(u,t  —  1)  to  the  destination  0.  At  time  t: 

1.  A  given  node  ut  is  activated 

2.  Node  ut  updates  its  path  to  be  the  most  preferred  and  available 
path  which  is  best(choices(7r(t  —  l),ut,t—  1)).  Therefore,  7r (ut,t)  = 
best(choices(7r(t  —  1  ),ut,t  —  1)). 

3.  All  other  nodes  leave  their  paths  unchanged.  Therefore,  if  v  £ 
V  —  {«*},  then  n(v,  t )  =  7r(v,  t  —  1) 

Figure  7.  The  SNASM  Routing  Protocol  Dynamics 


We  may  now  define  safety  in  terms  of  SNASM. 

Definition:  Safe  (SNASM)  An  instance  of  the  stable  paths  problem  is  safe 
(SNASM)  if  for  any  initial  path  assignment  7r(0)  and  any  single  node  fair  activation 
sequence  there  exists  a  finite  T  such  that  7r (t)  =  i r(T)  for  all  t  >  T.  In 

Chapter  4  we  show  that  if  an  instance  of  SPP  is  safe  (SNASM)  this  does  not  imply 
that  it  is  safe  (SPVP). 

3.  Multiple  Node  Activation  Sequence  Model 

Gao  and  Rexford  also  proposed  a  BGP  model  in  which  nodes  are  activated, 
and  receive  the  highest  ranked  path  available.  However,  in  their  model,  multiple 
nodes  may  be  activated  simulatenously.  Gao  and  Rexford  also  described  rankings 
and  filterings  in  terms  of  their  own  framework.  However,  we  will  write  the  multiple 
node  activation  sequence  model  (MNASM)  in  terms  of  the  stable  paths  problem. 

Definition:  Fair  Multiple  Node  Activation  Sequence  A  sequence  of 
sets  of  nodes  to  =  Lfi,  t/2,  U3...  is  said  to  be  a  fair  multiple  node  activation  sequence  if 
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each  node  v  E  V  appears  infinitely  often  in  the  sequence  as  the  element  of  some  set 

uk  c  V. 


MNASM  Routing  Protocol  Dynamics 

At  time  t  —  1,  the  current  path  assignment  is  7r(t  —  1).  Each  node  u  has 
currently  selected  path  n(u,t  —  1)  to  the  destination  0.  At  time  t: 

1.  A  Set  of  nodes  Ut  is  activated 

2.  Each  node  v  G  Ut  updates  its  path  to  be  the  most  preferred  and 
available  path  which  is  best(choices(7r(f  —  1  ),v,t  —  1)).  Therefore, 
if  v  e  Ut,  then  n (v,t)  =  best(choices(7r(f  —  1  ),v,t  —  1)). 

3.  All  other  nodes  V  —  Ut  leave  their  paths  unchanged.  Therefore,  if 
w  G  V  —  Ut,  then  7 r(w,  t)  =  n(iu,  t  —  1). 

Figure  8.  The  MNASM  Routing  Protocol  Dynamics 


Figure  8  presents  the  Multiple  Node  Activation  Sequence  Model.  Time  is 
modeled  discretely.  The  model  uses  a  fair  multiple  node  activation  sequence,  so  each 
node  is  activated  infinitely  often.  The  model  begins  with  a  path  assignment  7r(0). 
At  each  time  t.  a  set  of  nodes  Ut  are  activated.  This  corresponds  to  each  node  in 
Ut  instantaneously  and  simultaneously  receiving  the  path  assignments  at  time  t  —  1 
from  all  other  nodes.  Each  node  in  Ut  will  then  update  its  current  path  assignment  to 
the  highest  ranked  available  path.  Note  that  this  model  differs  from  SPVP  because 
it  does  not  allow  for  messages  from  different  neighbors  to  arrive  and  be  processed  in 
different  orders.  However,  it  can  model  the  possibility  that  the  routing  information 
at  a  given  node  is  not  current. 

We  can  now  define  safety  in  terms  of  the  multiple  node  activation  sequence 

model. 

Definition:  Safe  (MNASM)  An  instance  of  the  stable  paths  problem  is  safe 
(MNASM)  if  for  any  initial  path  assignment  7r(0)  and  multiple  node  fair  activation 
sequence  U\,  U2, ...,  there  exists  a  finite  T  such  that  7r(f)  =  7 r(T)  for  all  t  >  T. 
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4.  Comparison  of  Models  to  BGP 

For  several  reasons,  the  simple  path  vector  protocol  most  accurately  models 
BGP  behavior.  First,  RFC  4271  specihes  that  only  one  UPDATE  message  may 
be  processed  at  any  given  time.  Second,  while  MNASM  can  model  multiple  nodes 
receiving  update  messages  simultaneously,  we  will  show  in  Chapter  IV  that  SPVP 
can  match  any  path  assignment  reached  by  MNASM. 

However,  MNASM  is  a  much  simpler  model  to  conduct  proofs  on  because  one 
does  not  need  to  keep  track  of  the  state  of  message  queues.  In  Chapter  V,  we  will 
use  MNASM  for  the  proof  of  the  main  theorem  of  this  paper. 

E.  ROBUSTNESS 

Definition:  Robustness  An  instance  of  SPP  is  robust  (MODEL)  if  and  only 
if  that  instance  and  every  subinstance  is  uniquely  solvable  and  safe  (MODEL). 

For  this  paper,  if  no  model  is  specified,  we  take  robust  to  mean  robust  (MNASM). 

F.  DISPUTE  WHEELS 

The  concept  of  dispute  wheels  was  first  introduced  by  Griffin  and  Wilfong 
[Ref.  12].  A  dispute  wheel  is  a  sequence  of  nodes  and  paths  that  represent  mutually 
conflicting  policies  due  to  rankings.  These  mutually  conflicting  paths  may  cause  an 
instance  of  SPP  to  be  unsolvable  or  give  rise  to  permanent  oscillations,  making  the 
instance  not  safe. 

Formally,  a  dispute  wheel ,  n  =  (U ,  Q ,  R),  of  size  A;  is  a  sequence  of  nodes 
U  =  Uq,  Ui,  ...Wfc-i,  and  sequences  of  nonempty  paths  Q  =  Q0,  Qi,  ...Qk-i  and  R  = 
Rq,  Ri,  ...Rk- 1,  such  that  for  each  0  <  i  <  k  —  1  the  following  hold  true: 

1.  Ri  is  a  path  from  iq  to  ui+ 1 

2.  Qi  e  PUi 

3.  RiQi+l  e  Pu" 

4.  XUi(Qi)  <  XUi(RiQi+i) 
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Paths  of  the  form  Qi  are  often  described  as  spoke  paths.  Paths  of  the  form 
Ri  are  often  described  as  rim  paths.  Rule  1  specifies  the  form  of  a  rim  path.  Rule  2 
specifies  that  that  that  a  spoke  path  must  be  available  from  the  originating  node,  thus 
assuring  that  this  path  can  be  assigned  to  the  originating  node.  Rule  3  specifies  that 
the  combined  rim  path  and  spoke  path  must  be  available  from  the  originating  node, 
thus  assuring  that  this  path  can  be  assigned  to  the  originating  node.  Rule  4  stipulates 
the  preference  for  the  combined  rim  path  and  spoke  path,  over  a  spoke  path.  Because 
every  node  in  the  sequence  U  has  this  property,  the  policies  are  mutually  conflicting. 

Figure  9  presents  a  generalized  dispute  wheel.  The  next  section  presents  spe¬ 
cific  instances  of  SPP  and  examples  of  their  dispute  wheels. 


Figure  9.  A  Generalized  Dispute  Wheel 

As  discussed  above,  a  dispute  wheel  represents  a  set  of  mutually  conflicting 
rankings  for  some  nodes.  In  BGP,  this  would  represent  a  set  of  mutually  conflicting 
policies.  Griffin  and  Wilfong  proved  several  theorems  about  instances  of  SPP  that  do 
not  contain  dispute  wheels  [Ref.  12],  To  summarize,  they  proved  that  if  an  instance 
of  SPP,  S  does  not  contain  a  dispute  wheel,  then  S  is  uniquely  solvable,  safe  (SPVP), 
and  robust. 

Theorem  V.4  from  [Ref.  12]  1.  If  the  stable  paths  problem  S  has  no  dispute 
wheel,  then  S  has  a  unique  solution. 

Theorem  V.9  from  [Ref.  12]  1.  If  S  has  no  dispute  wheel,  then  S  is  safe 
(SPVP). 
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Theorem  V.10  from  [Ref.  12]  1.  Let  S  be  an  instance  of  the  stable  paths 
problem.  If  S  has  no  dispute  wheel,  then  S  is  robust  (SPVP). 

Once  Griffin  and  Wilfong  have  presented  dispute  wheels,  they  describe  one 
set  of  constraints  that  can  prevent  dispute  wheels.  They  show  that  any  instance  of 
SPP  that  uses  route  filtering  alone,  and  ranks  paths  based  only  on  hop  count  can  not 
contain  a  dispute  wheel.  If  these  constraints  are  followed  for  the  stable  paths  problem 
S,  then  S  is  guaranteed  to  be  robust. 


G.  INTERESTING  INSTANCES  OF  SPP 

1.  Solvable,  but  not  Safe  (SNASM  or  MNASM  or 
SPVP) 

In  Figure  10,  we  present  an  instance  of  SPP  that  is  solvable,  but  not  safe 
(SPVP  or  MNASM).  We  call  this  instance  “NEXT.”  NEXT  has  three  solutions.  In 
one  solution,  n  —  (1  0)(2  3  1  0)(3  1  0).  In  the  second  solution,  n  =  (1  2  0)(2  0)(3  1  2  0). 
In  the  third  solution,  n  =  (1  2  3  0)  (2  3  0)  (3  0). 


Figure  10.  SPP  Instance  NEXT 

Despite  having  three  solutions,  there  is  an  initial  path  assignment  and  fair 
multiple  node  activation  sequence  that  is  not  safe  (SNASM  or  MNASM).  Consider 
the  initial  path  assignment  7r(0)  =  (1  0)(2  0)(3  1  0).  Table  I  gives  an  unsafe  fair 
activation  sequence  for  NEXT.  This  activation  sequence  could  consist  of  either  single 
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nodes  or  singleton  sets,  so  both  activation  models,  SNASM  and  MNASM  apply  to  it. 
We  can  see  that  this  sequence  of  path  assignments  could  repeat  indefinitely  under  a 
fair  activation  sequence  because  the  path  assignment  at  time  0  is  the  same  at  time 
6,  and  all  nodes  are  activated  at  least  once  in  between.  We  claim  that  NEXT  is  also 
not  safe  (SPVP).  In  Chapter  IV  we  prove  why  this  is  true. 


time  i 

7T(z) 

0 

(1  0)  (2  0)  (3  1  0) 

1 

(1  2  0)  (2  0)  (3  1  0) 

2 

(1  2  0)  (2  3  1  0)(3  1  0) 

3 

((1  2  0)  (2  3  1  0)  (3  1  2  0) 

4 

(1  0)  (2  3  1  0)  (3  1  2  0) 

5 

(1  0)  (2  0)(3  1  2  0) 

6 

(1  0)  (2  0)  (3  1  0) 

Table  I.  Path  Assignments  of  NEXT.  If  a  path  assignment  is  underlined,  that  node 
has  been  activated  at  that  time. 


2.  Uniquely  Solvable,  but  Not  Safe  (MNASM) 

In  Figure  11,  we  present  an  instance  of  SPP  that  is  uniquely  solvable,  but  not 
safe  (MNASM).  The  instance  has  the  unique  solution  7r  =  (1  3  0)(2  0)(3  0)(4  3  0). 


Figure  11.  An  Instance  of  SPP  that  is  Uniquely  Solvable,  but  Not  Safe  (Naughty 
Gadget  from  [Ref.  12]  ) 
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Despite  this  unique  solution,  the  instance  is  not  safe.  Consider  the  initial  path 
assignment  tt  —  (1  0)(2  0)(3  4  2  0)(4  2  0).  Table  II  gives  an  unsafe  sequence  of  path 
assignments  that  may  be  repeated  indefinitely. 


step 

7 r 

0 

(1  0)  (2  0)  (3  4  2  0)  (4  2  0) 

1 

(1  0)  (2  1  0)  (3  4  2  0)  (4  2  0) 

2 

(1  0)  (2  1  0)  (3  4  2  0)  e 

3 

(1  0)  (2  1  0)  (3  0)  e 

4 

(1  0)  (2  1  0)  (3  0)  (4  3  0) 

5 

(1  3  0)  (2  1  0)  (3  0)  (4  3  0) 

6 

(1  3  0)  (2  0)  (3  0)  (4  3  0) 

7 

(1  3  0)  (2  0)  (3  0)  (4  2  0) 

8 

(1  3  0)  (2  0)  (3  4  2  0)  (4  2  0) 

9 

(1  0)  (2  0)  (3  4  2  0)  (4  2  0) 

Table  II.  An  Unsafe  Sequence  of  Path  Assignments  for  NAUGHTY  GADGET  from 
[Ref.  12],  If  a  path  assignment  is  underlined,  that  node  has  been  activated  at  that 
time. 


3.  Categories 

In  previous  sections,  we  defined  some  possible  properties  of  instances  of  SPP 
such  as  robustness,  unique  solvability,  and  safety.  We  would  like  to  categorize  these 
properties  in  relation  to  one  another.  Figure  12  shows  how  properties  of  an  SPP 
instance  relate  to  one  another,  over  the  space  of  all  SPP  instances.  In  this  diagram, 
we  assume  all  definitions  of  safety  and  robustness  correspond  to  the  same  model.  In 
Chapter  IV,  we  will  discuss  in  more  detail  how  these  definitions  are  related.  Note 
that  the  absence  of  a  dispute  wheel  is  not  a  sufficient  and  necessary  condition  for 
robustness.  In  Chapter  V,  we  will  introduce  an  instance  of  SPP  that  is  robust,  but 
has  a  dispute  wheel.  Also  note  that  safety,  implies  solvability,  because  for  an  instance 
of  SPP  to  be  safe  there  must  exist  a  solution. 
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Figure  12.  Properties  of  SPP  Instances  over  the  Space  of  All  SPP  Instances 

H.  HIERARCHICAL  BGP 

Gao  and  Rexford  [Ref.  8]  introduced  conditions  on  filtering,  ranking,  and 
topology  that  guarantee  the  convergence  of  BGP.  They  noted  that  every  eBGP  ses¬ 
sion  should  define  an  interorganizational  relationship  between  the  two  connected 
ASes.  They  limited  the  possible  relationships  to  only  peer-to-peer  relationships  and 
customer-provider  relationships.  Therefore,  given  an  AS,  u,  a  neighbor  w  must  be¬ 
long  to  the  set  of  providers,  provider (u)]  the  set  of  customers  customer  (u)]  or  the 
set  of  peers,  peer(u).  Note  that  Gao’s  definition  of  peer{u)  is  much  different  than 
Griffin’s  definition  of  peers(u).  In  Gao’s  definition,  a  neighbor  iv  €  peer{u)  will  follow 
strict  guidelines  that  will  be  discussed  below.  In  Griffin’s  definition  peers(u)  is  all  the 
neighbors  of  u,  so  we  have  peers(u)  =  provider{u )  [J  peer(u)  [j  customer (u). 

Gao  and  Rexford  introduce  one  topological  constraint.  There  can  be  no  cycle 
of  provider-customer  relationships.  More  precisely  let  the  provider-to- customer  graph 
be  a  subgraph  generated  where  the  only  edges  are  directed  from  provider  to  customer. 
This  resulting  subgraph  should  be  acyclic  (or  a  DAG). 

Gao  and  Rexford  introduce  a  number  of  filtering  policies  that  reflect  the  real 
world  configuration  of  ASes.  These  policies  reflect  the  idea  than  an  ISP  should  not 
advertise  routes  for  traffic  without  financial  benefit.  These  rules  are  summarized  as 
the  following  policies: 
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•  Exporting  to  a  customer:  In  exchanging  routing  information  with  a  cus¬ 
tomer,  an  AS  can  export  its  routes,  as  well  as  routes  learned  from  its  providers 
and  peers. 

•  Exporting  to  a  provider  or  peer:  In  exchanging  routing  information  with 
a  provider  or  peer,  an  AS  can  export  its  routes  and  the  routes  of  its  customers, 
but  it  can  not  export  routes  learned  from  other  providers  or  peers. 

Gao  and  Rexford  also  introduce  a  number  of  guidelines  on  the  ranking  function 

of  individual  ASes.  A  system  of  ASes  is  said  to  meet  Guideline  A  [Ref.  8]  if  every  AS 

prefers  a  route  via  a  customer  over  a  route  via  a  provider  or  peer.  Formally,  let  S  be 

an  instance  of  SPP.  For  all  u  G  V,  for  all  P±,  P2  G  Vu  where  P\  =  (ux... 0)  and  P2  = 

{uy... 0),  if  x  G  customer{u)  and  y  G  provider{u )  (J peer(u)  then  Xu(Pi)  >  XU(P2). 

Gao  and  Rexford  proved  that  a  system  of  ASes  which  follows  Guideline  A  has 

a  stable  state  and  is  safe  under  the  Multiple  Node  Activation  Sequence  Model.  We 

use  a  different  proof  to  show  that  any  system  of  ASes  which  follows  Guideline  A  can 

not  contain  a  dispute  wheel  and  is  robust. 

Theorem  III.l.  If  an  instance  of  SPP  meets  the  exporting  policies,  the  topo¬ 
logical  constraint  and  Guideline  A  from  [Ref.  8],  then  the  instance  of  SPP  can’t 
contain  a  dispute  wheel  and  is  robust. 

Proof.  We  use  proof  by  contradiction.  Suppose  an  instance  of  SPP  meets  the 
exporting  policies,  the  topological  constraint,  and  Guideline  A  [Ref.  8]  and  has  a 
dispute  wheel.  Let  the  dispute  wheel  U  —  (U ,  Q ,  R)  have  size  k.  For  each  Qj  G  Q 
of  length  m,  let  Qj  be  the  path  Qj  =  ■■■Qj, m),  where  =  uj  and  qj,m.  =  0. 

For  each  Rj  G  R  of  length  n,  let  R,  be  the  path  Rj  =  {r]j)r]Ar]/2...r,jn)1  where 
G,o  —  uj  and  rj}Tl  =  u3+\.  Due  to  the  export  filters  on  each  AS  (or  node),  for  all  paths 
Q3  of  size  771,  we  must  have  qjj-i  G  customer {q3  i)  for  all  1  <  i  <  m.  If  this  was 
not  the  case,  then  the  path  Rj_iQj  would  not  be  available  to  Uj_ i,  Rj-iQj  ^  Pu'i~1 
due  to  export  Liters.  Therefore,  because  q3)o  G  customer (gyi),  we  must  also  have 
Tjfi  G  customer(rjti ),  otherwise  the  route  Rj-iQj  would  not  be  preferred  to  Qj-i  due 
to  the  fact  that  its  first  hop  would  be  a  provider  or  a  peer.  Also  due  to  the  export 
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filters  at  each  node,  for  all  paths  Rj  of  size  n,  we  must  have  rhi_  \  e  customer {rj^ 
for  all  1  <  i  <  n.  If  this  was  not  the  case  then  the  path  RjQj+ 1  would  not  be 
available  to  node  Uj.  We  have  now  formed  a  cycle  of  customer- provider  relationships 
along  the  path  (R0Ri...Rk).  However,  this  contradicts  the  topological  constraint  that 
the  provider-to-customer  graph  is  acyclic.  Therefore,  we  have  a  contradiction.  We 
have  shown  by  contradiction  that  an  instance  of  SPP  can  never  meet  the  exporting 
policies,  the  topological  constraint,  and  Guideline  A  [Ref.  8]  and  have  a  dispute 
wheel.  Therefore,  If  an  instance  of  SPP  meets  the  exporting  policies,  the  topological 
constraint  and  Guideline  A  [Ref.  8],  then  the  instance  of  SPP  can’t  contain  a  dispute 
wheel.  The  instance  of  SPP  must  also  be  robust,  because  it  contains  no  dispute 
wheels. 

□ 

Gao  and  Rexford  developed  this  model  further  to  allow  for  a  back-up  relation¬ 
ship  between  neighboring  ASes. 

I.  CLASS-BASED  PATH- VECTOR  SYSTEMS 

Griffin,  Jaggard,  and  Ramachandran  introduced  a  much  more  general  form 
of  Gao  and  Rexford’s  model,  called  the  class-based  path- vector  system  [Ref.  11], 
Jaggard  and  Ramachandran  presented  a  generalized  framework  that  can  be  used  to 
describe  any  BGP  system  where  the  filtering  (also  called  scoping)  rules  and  ranking 
rules  are  based  upon  the  relationships  between  classes  of  ASes. 

Informally,  a  path-vector  system  describes  some  of  the  low  level  characteristics 
of  a  path-vector  protocol.  A  path-vector  system  describes  the  possible  destinations, 
paths  to  destinations  which  may  be  exchanged,  rankings  for  available  paths,  some 
basic  local  import/  export  constraints,  and  some  basic  import  /  export  transformation 
rules.  Rankings  for  available  paths  may  be  specified  similar  to  the  way  RFC  4271 
ranks  paths  in  BGP,  or  by  use  of  other  metrics  such  as  shortest  hop  count  alone. 
Basic  local  constraints  make  sure  that  paths  known  at  a  given  node  satisfy  certain 
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properties.  For  instance,  paths  should  only  be  imported  if  the  destination  is  a  possible 
routing  destination.  Basic  /  import  export  rules  may  be  configured  to  exclude  paths 
which  contain  loops  or  perform  other,  less  mundane  actions.  A  path-vector  system 
can  be  used  to  describe  most  characteristics  of  BGP.  However,  a  path-vector  system 
does  not  describe  exactly  how  messages  are  exchanged  between  nodes. 

Griffin,  Jaggard,  and  Ramachandran  also  define  a  policy  language  to  capture 
high  level  characteristics  of  a  system.  For  BGP,  a  policy  language  may  describe 
whether  a  path  is  given  a  specified  LOCAL_PREF  attribute  when  the  path  is  imported 
from  specified  neighbors.  Together,  a  path-vector  system  and  policy  language  may 
be  used  to  describe  the  stable  paths  problem. 

The  class-based  path- vector  systems  are  a  set  of  policy  languages  which  meet 
some  general  constraints.  These  constraints  are  formed  using  matrices.  First,  every 
class-based  path- vector  system  has  a  set  of  classes,  such  as  “customer”  or  “provider” . 
The  cross-class  matrix  describes  which  relationships  may  occur  and  row/column 
numbers  correspond  to  specific  classes.  Each  row  and  column  in  this  matrix  has 
exactly  one  “1”  and  all  other  entries  are  “0.”  This  matrix  may  describe  facts  such 
as  “customer-provider  relationships  are  allowed”  or  “customer-peer  relationships  are 
not  allowed.”  The  preference  matrix  describes  some  ranking  rules  for  different  classes, 
such  as  “prefer  all  paths  received  from  customers  to  all  paths  received  from  providers.” 
The  level  matrix  describes  the  scoping  rules  such  as  “export  all  routes  learned  from 
a  customer  to  a  provider.”  These  preference  matrix  and  level  matrix  can  also  be  used 
to  describe  hierarchical  properties  of  BGP.  For  instance,  depending  on  whether  a 
relationship  is  with  a  tier  1  or  tier  2  peer  different  exporting  rules  may  be  specified. 

Jaggard  and  Ramachandran  continued  work  on  class-based  path- vector  sys¬ 
tems  by  giving  a  much  more  general  form  of  Theorem  III.  1 .  In  their  paper  they  give 
an  exact  condition  for  dispute  wheel  creation  based  upon  the  particular  relationships, 
scoping  rules,  and  ranking  rules  of  a  particular  system,  as  well  as  global  constraints 
[Ref.  18].  This  exact  condition  is  still  stricter  than  a  necessary  and  sufficient  condi- 
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tion  to  guarantee  BGP  robustness,  because  as  we  have  seen,  some  instances  of  SPP 
(and  systems  of  ASes)  may  have  dispute  wheels  and  still  be  robust. 
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IV. 


COMPARISON  OF  BGP  MODELS 


In  the  previous  chapter  we  introduced  three  different  models  of  BGP  behavior; 
the  simple  path  vector  protocol  (SPVP),  the  single  node  activation  sequence  model 
(SNASM),  and  the  multiple  node  activation  sequence  model  (MNASM).  Because  all 
models  are  expressed  in  terms  of  the  stable  paths  problem,  solvability  and  unique 
solvability  is  equivalent  between  the  three  different  models.  However,  in  this  chapter 
we  investigate  two  other  issues.  First,  we  investigate  whether  any  sequence  of  path 
assignments  given  by  one  model  can  match  another  model.  Second,  we  investigate 
whether  safety  in  one  model  implies  safety  in  another  model. 

A.  MATCHING  PATH  ASSIGNMENTS 

We  investigate  whether  any  sequence  of  path  assignments  given  by  one  model 
can  be  matched  by  another  model.  Informally,  we  describe  matching  as  the  ability 
of  one  model  to  begin  with  the  same  path  assignment  as  another  model,  and  reach 
all  possible  subsequent  path  assignments  for  the  other  model.  We  allow  intermediate 
path  assignments  to  be  taken  between  equal  path  assignments.  We  say  that  the 
sequence  of  path  assignments  cu  =  ni (0),  711(1),  7Ti(2), ...  matches  the  sequence  of  path 
assignments  a  =  7T2(0),  7t2(1),  7t2(2),  ...  if  there  exists  a  subsequence  of  uj  that  is  equal 
to  a. 


Figure  13.  An  Instance  of  SPP  That  Shows  MNASM  Does  Not  Match  SPVP 
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Definition:  Matching  of  Models  We  say  that  BGP  model  A  matches  BGP 
model  B,  if  for  any  sequence  of  path  assignments  given  by  model  B  denoted  by  a, 
there  exists  a  sequence  of  path  assignments  given  by  BGP  model  A  that  matches  a. 

We  proceed  to  our  negative  results  first. 

Theorem  IV. 1  (MNASM  does  not  match  SPVP).  The  multiple  node  activa¬ 
tion  sequence  model  does  not  match  the  simple  path  vector  protocol 

Proof.  Consider  the  following  counterexample  presented  in  Figure  13.  Let 
c o  =  7rw(0),7rw(l),7rw(2), ...  be  the  sequence  of  path  assignments  given  by  SPVP  that 
we  will  show  can  not  be  matched  by  the  multiple  node  activation  sequence  model.  As 
usual,  the  path  assignment  at  node  u  at  time  t  is  denoted  by  ^(m,  £).  We  induce  the 
initial  state  as  follows,  let  7iy,(3,  0)  =  (3  0)  and  let  nodes  1  and  2  have  the  empty  path 
assignment.  For  each  u  G  V  and  w  G  peers(u),  let  mq(«  4=  w)  be  a  message  informing 
u  of  7r(zc,0).  As  depicted  in  Figure  14,  there  is  a  sequence  of  path  assignments  in 
SPVP  that  gives  a  final,  stable  path  assignment  —  (1  0)(2  1  0)(3  2  1  0).  This  is 
achieved  by  processing  messages  in  the  following  order.  At  t  =  1,  node  1  processes 
mq(l  0) [1]  and  changes  its  path  such  that  7r(l,  1)  =  (1  0).  At  t  —  2,  node  2 
processes  mq(2  1) [1]  and  keeps  the  empty  path  assignment.  At  t  =  3,  node  2 
processes  mq(2  1) [1]  =  (1  0)  and  changes  its  path  assignment  to  7r(2,3)  =  (2  1 
0).  At  t  =  4,  5,6,  7,  node  3  processes  mq(3  -<=  l)[l],mq(3  l)[2],mq(3  4=  0)[l],and 

mq(3  -<=  2) [1]  and  does  not  change  its  path  assignment.  At  t  —  8,  node  3  processes 
mq(3  2) [1]  =  (2  10)  and  changes  its  path  assignment  to  7r(3,  8)  =  (321  0). 

However,  for  this  same  initial  path  assignment,  the  multiple  node  activation 
sequence  model  can  not  reach  these  subsequent  path  assignments.  This  is  because 
when  any  node  is  activated,  it  receives  the  highest  ranked  paths.  Therefore,  only 
node  1  can  change  path  assignments  and  will  change  its  path  to  (1  3  0).  This  is  a 
stable  path  assignment,  and  no  future  changes  can  occur. 

Therefore,  there  exists  a  sequence  of  path  assignments  given  by  SPVP  that 
can’t  be  matched  by  any  sequence  of  path  assignments  given  by  MNASM 
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Figure  14.  A  Sequence  of  Path  Assignments  Given  by  SPVP  for  Figure  13 

We  have  shown  that  the  multiple  node  activation  sequence  model  does  not 
match  the  simple  path  vector  protocol. 

□ 

However,  we  would  like  to  know  whether  SPVP  matches  the  multiple  node 
path  vector  protocol. 

Theorem  IV.2  (SPVP  matches  MNASM).  The  simple  path  vector  protocol 
matches  the  multiple  node  activation  model. 

Proof.  Let  S  be  an  instance  of  the  stable  paths  problem.  Let  u  =  Ui,  U2, ... 
be  any  fair  multiple  node  activation  sequence  and  let  a  =  71^(0),  71^(1),  71^(2)  be  the 
sequence  of  path  assignments  for  uj  given  by  the  MNASM.  We  would  like  to  show 
that  there  exists  a  sequence  of  path  assignments  given  by  SPVP  that  matches  u.  Let 
6  =  7Tspyp(0),  7T5pyp(l),  7Tspyp(2), ...  be  the  subsequence  of  path  assignments  given 
by  SPVP  we  are  trying  to  form.  We  would  like  to  show  that  there  exists  an  initial 
state  for  SPVP  and  ordering  of  message  receipts  such  that  n spypif)  —  A, j(i)  for  all 
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i  >  0.  Let  the  initial  state  be  induced  by  71^(0)  such  that  7iq,(0)  =  Tispvpi^)-,  each 
rib-in(u  -4=  w)  =  e,  and  each  message  queue  mq(«  w)  =  7T^(w,  0). 

Let  X(i )  be  the  induction  predicate  that  after  the  ith  element  of  the  subse¬ 
quence  6  has  been  formed,  7r^(?.)  =  n  spvp(i)  and  for  all  nodes  u  G  V,  if  best  (choices^  (i),w,i)) 
has  a  next  hop  of  w  and  7 Tspvp(u,i)  ^  best(choices(7r(i),w,i))  either  there  is  message 
in  the  queue  mq(«  -4=  w)  that  informs  u  of  the  path  b es t ( choices (7i(i).u,i  j)  OR  rib- 
in (u  4=  w)  has  this  path  stored  already. 

Base  Case.  The  predicate  A'(0)  holds  true  because  our  initial  state  has  those 
properties. 

Induction  Step.  Suppose  X(i)  is  true.  Under  MNASM,  a  set  of  nodes  Uji+\ 
will  be  activated  at  time  i  + 1.  For  each  u  G  U+ 1,  we  will  never  process  messages  that 
have  been  generated  since  the  path  assignment  nspvp{i)  was  reached,  because  this 
may  cause  a  node  to  take  a  path  assignment  other  than  the  one  we  would  like  to  be 
taken  at  7iq,(i).  However,  we  process  all  other  messages  in  all  queues,  in  any  arbitrary 
order.  This  will  give  u  an  exact  picture  of  what  neighboring  path  assignments  were 
under  7 Tspvp(i),  and  guarantees  that  the  best(choices(7r(f  +  1  ),v,i  +  1))  will  be  the 
final  path  selected.  Once  this  has  been  completed  for  each  node  u  G  Ut+i,  we  have 
created  a  path  assignment  7 Tspvpi}  +  1)  =  7]q,(i  +  f)  Now,  suppose  any  other  node 
v  is  no  longer  assigned  the  path  best  (choices  (n(i  +  1  ),v,i  +  1))  and  this  path’s  first 
next  hop  is  w.  Suppose  the  node  v  (jL  Ui  .  It  must  still  either  have  a  message  in 
the  queue  mq(u  4=  w)  informing  v  of  that  path  or  rib-in(u  4=  w)  has  this  path. 
Suppose  the  node  v  was  activated  in  the  set  Ui.  There  are  two  possible  cases,  or  a 
combination  of  both.  In  the  first  case,  a  neighboring  node  x  G  Ui  was  activated  and 
changed  its  path  assignment  so  now,  v  has  an  even  higher  ranked  path  available  in 
choices ( 7T (f  +  1  ),v,i  +  f)  that  was  not  available  in  choices(7r(z),n,z).  However  this  path 
should  have  been  advertised,  so  there  must  be  a  message  in  the  queue  mq(«  4=  x) 
informing  v  of  this  path.  Otherwise,  suppose  the  path  7 tspvp(v,i  +  1)  has  been 
withdrawn.  The  new  path  best  (choices  (77(7  +  1  ),v,i  +  1))  must  either  be  contained  in 
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a  rib  —  in  or  have  been  advertised  by  the  withdrawal.  Therefore  X[i)  1) 

By  the  principle  of  induction,  we’ve  shown  that  the  path  assignments  generated 
under  MNASM  can  be  generated  by  taking  a  subsequence  of  path  assignments  under 
SPVP. 

Finally,  we  must  check  that  the  processing  of  messages  is  fair ;  every  message 
will  eventually  be  processed.  The  processing  of  messages  is  fair,  because  every  node 
is  activated  infinitely  often  under  MNASM,  and  each  time  a  node  is  activated,  all  old 
messages  will  processeded  before  the  last  pat-est  assignment  was  generated. 

□ 

Corollary  IV. 3  (SPVP  matches  SNASM).  By  a  similar  argument,  the  simple 
path  vector  protocol  matches  the  single  node  activation  sequence  model. 

Theorem  IV.4  (MNASM  matches  SNASM).  The  multiple  Jiode  activation 
sequence  model  matches  the  single  node  activation  sequence  model 

Proof.  Let  to  =  u0,  U\,  u2, ...  be  any  fair  single  node  activation  sequence.  We 
form  a  fair  multiple  node  activation  sequence  c vl  by  simply  taking  one  element  subsets 
such  that  uil  =  {«o},  {^i } ,  {^2}, ...  .  The  path  assignment  for  u  will  exactly  equal  the 
path  assignment  for  co\  because  nodes  are  activated  identically  under  both  models, 
and  the  same  node  is  activate  at  each  time. 

□ 

Figure  15  depicts  the  result  of  this  section.  We  consider  the  space  of  instances 
of  SPP  and  initial  path  assignments.  The  intersection  of  two  models  describes  an 
instance  of  SPP  and  initial  path  path  assignments  for  both  models  that  match  each 
other  for  any  possible  sequence  of  path  assignments.  Likewise,  the  places  where  model 
A  does  not  intersect  with  model  B  describes  an  instance  of  SPP  and  initial  path 
assignment  that  results  in  a  sequence  of  path  assignments  that  can  not  be  matched 
by  model  A. 
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Simple  Path  Vector  Protocol 

Multiple  Node  Activation 
Sequence  Model 


Single  Node  Activation 
Sequence  Model 


Figure  15.  How  Models  Match  Each  Other  over  the  Space  of  Instances  of  SPP  and 
Initial  Path  Assignments 

B.  COMPARISON  OF  SAFETY 

In  the  preceeding  section,  we  investigated  whether  different  models  of  BGP 
match  each  other.  We  can  use  these  results  to  show  that  safety  as  defined  in  one 
model  can  imply  safety  in  another  model. 

Theorem  IV. 5.  Let  S  be  an  instance  of  the  stable  paths  problem.  If  S  is  not 
safe  (MNASM),  then  S  is  not  safe  (SPVP). 

Proof.  Suppose  S  is  not  safe  (MNASM).  Then  there  exists  at  least  one  fair 
multiple  node  activation  sequence,  such  that  there  is  not  finite  time  T  where  the 
7 r(T)  =  7T (t)  for  all  t  >  T.  By  Theorem  IV. 2,  there  is  an  ordering  of  message 
processing  such  that  SPVP  will  have  a  subsequence  of  path  assignments  with  the 
exact  same  path  assignments.  Therefore,  the  message  queues  can  never  empty  and  S 
is  not  safe  (SPVP) 

□ 

Corollary  IV. 6  (Safe  (SPVP)  Safe  (MNASM)).  By  the  contrapositive 
of  Theorem  IV. 5,  If  S  is  an  instance  of  SPP  that  is  safe  (SPVP),  then  S  is  safe 
(MNASM) 

Theorem  IV. 7.  Let  S  be  an  instance  of  the  stable  paths  problem.  If  S  is  not 
safe  (SNASM),  then  S  is  not  safe  (MNASM) 


46 


Proof.  Suppose  S  is  not  safe  (SNASM).  Then  there  exists  some  fair  single 
node  activation  sequence  such  that  there  is  no  finite  time  T  where  where  the  7 r(T)  = 
7 v(t)  for  all  t  >  T.  By  Theorem  IV. 4,  we  can  form  a  fair  multiple  node  activation 
sequence  from  this  sequence  which  also  has  the  property  where  there  is  no  finite  time 
T  where  where  the  7 r(T)  =  tt (t)  for  all  t  >  T .  By  definition,  S  is  not  safe  (MNASM). 

□ 

Corollary  IV. 8  (Safe  (MNASM)  Safe  (SNASM)).  By  the  contrapositive 
of  Theorem  IV.  7,  If  S  is  an  instance  of  SPP  that  is  safe  (MNASM),  then  S  is  safe 
(SNASM) 

The  following  corollary  is  derived  from  applying  Corollaries  IV. 6  and  IV. 8. 

Corollary  IV. 9.  If  S  is  an  instance  of  SPP  that  is  safe  (SPVP),  then  it  is 
safe  (SNASM). 

Theorem  IV. 10  (Safe  (SNASM)  Safe  (MNASM)).  Let  S  be  an  instance 
of  SPP.  If  S  is  Safe  (MNASM),  then  this  does  not  imply  that  S  is  safe  (SNASM). 

Proof.  Consider  the  the  following  counterexample,  which  is  presented  in  Fig¬ 
ure  16.  This  instance  of  SPP  is  not  safe  (MNASM).  Let  S  have  the  initial  path 
assignment  7r(l,0)  =  (10)  and  7r(2,0)  =  (20).  This  routing  system  will  not  converge 
under  the  fair  multiple  node  activation  sequence,  {12},  {12},  {12}, ....  However,  S 
is  safe  (SNASM).  Given  any  fair  single  node  activation  sequence  and  initial  path 
assignment,  it  will  always  converge. 

□ 

Figure  17  depicts  the  result  of  this  section.  We  consider  the  space  to  be  the  set 
of  all  instances  of  the  stable  paths  problem.  We  are  not  sure  whether  Safe  (MNASM) 
Safe  (SPVP)  or  not.  It  is  possible  that  these  two  areas  are  exactly  equal. 

In  this  section  we  have  shown  that  models  of  BGP  do  not  necessary  have 
equivalent  definitions  of  safety,  and  that  some  path  assignments  of  some  models  can 
not  necessarily  be  matched  by  other  models.  These  results  have  important  conse¬ 
quences.  For  instance,  a  theorem  proved  about  robustness  using  one  model,  may  not 
necessarily  imply  robustness  for  other  models. 
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(2  1  0) 
(2  0) 


Figure  16.  Instance  of  SPP  for  Theorem  IV. 10.  DISAGREE  from  [Ref.  12] 


Safe  (SNASM) 


Safe(MNASM) 


Safe  (SPVP) 


Figure  17.  Safety  Between  Different  Models 
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V.  A  WEAKER  SUFFICIENT  CONDITION 

FOR  ROBUSTNESS 

A.  MOTIVATION 

There  have  been  several  proposals  to  guarantee  the  robustness  of  BGP.  In 
this  paper,  we  pursue  an  approach  that  relies  on  operational  guidelines  and  global 
constraints.  Griffin  et  al  showed  that  if  an  instance  of  SPP  does  not  have  a  dispute 
wheel,  the  instance  must  be  robust.  Unfortunately,  this  condition  is  too  strict;  there 
exist  instances  of  SPP  which  contain  dispute  wheels  but  are  robust.  Consider  the 
instance  of  SPP  in  Figure  18.  This  instance  of  SPP  is  robust,  but  contains  the 
dispute  wheel  in  Figure  19.  In  this  section,  we  give  a  weaker  sufficient  condition  for 
robustness.  Our  approach  focuses  on  determining  whether  the  subinstance  of  SPP 
generated  for  each  dispute  wheel  1)  is  robust  and  2)  has  the  property  such  that  for 
each  node  of  the  dispute  wheel,  all  possible  paths  are  contained  in  the  dispute  wheel. 


Figure  18.  An  Instance  of  SPP  that  has  a  Dispute  Wheel,  but  is  Robust 


Once  we  give  a  weaker  sufficient  condition  for  robustness,  we  investigate  how 
to  determine  whether  instances  of  SPP  are  robust,  despite  the  presence  of  one  or 
more  dispute  wheels.  We  focus  on  developing  new  global  and  local  constraints  that 
guarantee  robustness,  despite  the  presence  of  a  dispute  wheel. 
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Figure  19.  The  Dispute  Wheel  of  Figure  18 


B.  SUBINSTANCES  OF  SPP  FROM  DISPUTE  WHEELS 

Given  an  instance  of  SPP  S  =  (G,V,  A),  there  is  a  natural  way  to  derive 
subinstances  of  SPP  from  the  dispute  wheels  of  S.  Given  a  dispute  wheel  II  = 
(U ,  Q ,  R),  we  define  SPP(II)  =  (Gu,Vu,  An)  to  be  the  derived  instance  of  SPP 
from  II.  Let  the  graph  G*n  =  (Vfu-En)  have  the  property  where  In  contains  every 
vertex  that  appears  in  Q  and  R  and  En  contains  every  edge  (u  v)  G  E  such  that 
u,v  G  In  .  Let  the  set  of  available  paths  Pn  =  {P\P  G  V}  and  every  edge  in  the 
path  P  is  present  in  Eu  .  For  each  node  u,  we  will  denote  its  set  of  available  paths  as 
V1^.  Finally,  let  the  ranking  function  be  An  =  A,  but  modified  to  exclude  all  omitted 
paths. 

We  define  a  dispute  wheel  II  to  be  robust  if  SPP(II)  is  robust. 


C.  ALL  DISPUTE  WHEELS  ROBUST  IMPLIES  UNIQUELY 
SOLVABLE 

To  prove  than  an  instance  of  SPP  is  robust,  we  need  to  show  that  the  instance 
(and  every  subinstance)  of  SPP  is  uniquely  solvable. 

Theorem  V.l.  If  every  dispute  wheel  of  a  stable  paths  problem  is  robust  (or 
even  just  uniquely  solvable),  then  the  stable  paths  problem  is  uniquely  solvable. 

This  proof  closely  follows  the  proof  of  Theorem  V.4  [Ref.  12], 
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Proof.  We  use  proof  by  contradiction.  Let  S  be  an  instance  of  the  stable 
paths  problem.  Suppose  that  every  dispute  wheel  of  S  is  robust,  and  it  has  at  least 
two  distinct  solutions  7Ti  =  (Pi,  ...,P„_i)  and  n2  =  {Q i,  ...Qn-\).  As  discussed  above, 
every  solution  defines  a  tree  rooted  at  the  origin.  Let  Ti  and  T2  be  trees,  rooted  at 
the  origin  the  origin,  that  are  defined  by  7Ti  and  n2  respectively.  Given  a  graph  or 
component  G  let  V ( G )  and  E(G)  be  the  vertices  and  edges  of  the  graph  or  compo¬ 
nent  respectively.  Let  H  be  the  graph  (V,E(Ti)  fl  E(T2)).  Let  T  be  the  connected 
component  of  El  containing  the  origin.  Note  that  T  must  be  a  tree  because  it  is  a 
intersection  of  two  trees.  Every  edge  of  E[Ti  UT2)  not  contained  in  E(T )  =  {Tx  fl  T2) 
is  either  in  E{Ti  —T2)  or  E{T2  —  Ti).  We  say  an  edge  {uv)  is  entering  a  set  of  vertices 
V  if  exactly  one  of  the  nodes  {«,  u}  is  in  the  set  of  vertices  V.  Therefore,  every  edge 
of  Pi  U  T2  entering  V(T )  must  either  be  in  E(T\  —  T2 )  or  E{T2  —  T\). 


Figure  20.  Illustration  for  Theorem  V.l.  The  nodes  inside  the  dashed  circle  all 
represent  nodes  belonging  to  T.  Outside  the  circle  are  edges  and  nodes  in  Ti  and  T2. 
Dashed  Edges  are  in  T2.  Solid  edges  are  in  T\ 

We  now  construct  a  dispute  wheel.  Figure  20  visually  presents  the  dispute 
wheel  that  will  be  formed.  Note  that  because  the  two  solutions  are  unique,  Ti  ^  T2, 
the  set  of  vertices  V— V  (T)  must  be  nonempty  and  at  least  one  of  the  trees  has  an  edge 
entering  V (T).  Without  loss  of  generality,  consider  any  two  nodes  u  ,  v  in  Ti  such  that 
v  G  T  and  u  ^  T.  The  node  u  can  not  have  the  empty  path  assignment  in  7t2  because 
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it  can  not  prefer  the  empty  path  to  the  available  path  ( uv)Pv .  Therefore,  the  node  u 
must  also  belong  to  T2.  We  will  begin  to  construct  the  dispute  wheel  by  choosing  an 
edge  {woWo}  £  T\  such  that  u0  V(T)  and  v0  G  V(T).  As  discussed  above  u0  does 
have  a  path  to  the  origin  through  T2  which  must  be  of  the  form  Ro{u\V\)Qvl  that 
has  the  following  properties:  (i)  U\  qL  V(T)  and  V\  G  V(T)  (ii)  The  path  R0  is  a  path 
from  uq  to  U\  in  T2  and  contained  entirely  in  the  node  set  V  —  V(T)  (iii)  Finally,  R0 
must  have  a  length  of  at  least  one,  otherwise  one  of  the  paths  7r1(-u0)  or  tt2(u0)  would 
be  unstable.  This  process  is  repeated  at  node  U\  except  now  we  already  have  a  path 
directly  to  the  origin  for  T2  and  we  are  looking  for  a  path  to  the  origin  through  T\ . 
We  continue  alternating  and  searching  for  paths  in  this  fashion  until  we  eventually 
repeat  some  node,  which  without  loss  of  generality  is  uq.  We  must  eventually  repeat  a 
node  because  the  set  of  nodes  in  V  —  V (T)  is  finite  and  during  our  search  we  continue 
to  reach  a  new  node  each  time  unless  a  node  has  been  repeated. 

We  must  now  show  that  we  have  created  a  dispute  wheel.  Due  to  our  con¬ 
struction,  we  have  already  shown  all  the  properties  of  a  dispute  wheel  except  that  for 
each  i,  A Ui({uiVi)Qi)  <  XUi(Ri(ui+iVi+i)Qi+i).  To  show  this,  we  assume  without  loss 
of  generality  that  the  path  ( UiVi)Qi  is  contained  in  7\  .  Suppose  the  inequality  did 
not  hold.  Then  we  would  have  A Ui((uiVi)Qi)  >  \Ui(Ri[ui+iVi+i)Qi+i)  which  would 
mean  that  T2  should  have  preferred  the  same  path  to  T  and  that  T2  is  not  stable. 
But,  this  contradicts  our  assumption,  so  the  inequality  must  hold  and  we  must  have 
created  a  dispute  wheel  that  has  at  least  two  distinct  solutions. 

However,  the  dispute  wheel  must  have  a  unique  solution,  because  every  dispute 
wheel  of  S  is  robust.  Therefore,  we  have  a  contradiction.  We  have  used  indirect  proof 
to  show  that  if  every  dispute  wheel  of  an  instance  S  of  SPP  is  robust,  then  the  instance 
of  SPP  is  uniquely  solvable. 

□ 
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D.  ALL  DISPUTE  WHEELS  ROBUST  AND  COMPLETE 
IMPLIES  SAFETY 

To  prove  an  instance  of  SPP  is  robust,  we  must  also  show  that  the  instance 
(and  every  subinstance)  is  safe. 

Griffin  et  al.  gave  a  procedure  to  construct  a  dispute  wheel  given  an  unsafe 
instance  of  SPP.  We  will  use  a  similar  method  to  construct  a  dispute  wheel  that  is 
not  safe.  They  used  the  procedure  to  prove  the  following  theorem.  [Ref.  12]: 

Theorem  V.9  from  [Ref.  12]  2.  If  S  has  no  dispute  wheel,  then  S  is  safe 
(SPVP). 

1.  Selecting  an  Appropriate  Model 

In  the  previous  chapter,  we  compared  the  various  BGP  Models.  For  the  fol¬ 
lowing  proofs,  we  will  use  the  multiple  node  activation  sequence  model.  We  believe 
that  these  results  could  be  proved  differently  to  provide  similar  results  for  the  simple 
path  vector  protocol.  However,  for  the  remainder  of  the  chapter,  when  we  describe 
a  stable  paths  problem  to  be  “safe,”  we  specifically  mean  that  it  is  safe  (MNASM). 
Likewise,  when  we  describe  a  stable  paths  problem  to  be  “unsafe,”  we  mean  that  it 
is  not  safe  (MNASM). 

2.  Complete  Dispute  Wheels 

We  introduce  the  concept  of  a  complete  dispute  wheel.  We  will  show  that  if 
every  dispute  wheel  of  an  instance  of  SPP  is  complete  and  robust,  then  the  instance 
is  robust.  Much  of  the  following  notation  is  taken  from  [Ref.  12], 

Suppose  S  is  an  instance  of  the  stable  paths  problem  that  is  not  safe  (MNASM). 
For  some  initial  path  assignment  7r(0)  and  activation  sequence  a,  there  does  not  ex¬ 
ist  any  finite  time  T  such  that  the  path  assignment  does  not  change  after  time  T. 
However,  there  exist  some  nodes  that  do  not  change  their  paths  infinitely  often.  We 
define  the  set  of  nodes  C  to  be  the  nodes  that  do  not  change  their  path  assignment 
after  time  Tc.  We  define  the  set  O  to  be  the  set  of  nodes  that  change  their  paths 
infinitely  often.  For  each  node  u  G  V  we  define  values(<7,7r(0),it)  to  be  the  set  of  paths 
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that  u  adopts  infinitely  often.  Note  that  for  u  G  C,  values (cr,7r(0),-u)  will  be  a  one 
element  set,  equal  to  {n(tc,u)}. 

Suppose  P  is  a  path  of  the  form  (w0Wi...Wk).  We  define  P[wtw3\  as  the  sub¬ 
path  (iViWi+i...Wj).  Also,  we  define  P[l]  to  be  the  first  node  iu0. 

Let  S  be  an  unsafe  instance  of  SPP.  Let  U  be  the  set  of  all  nodes  such  that 
u  G  O  and  u  adopts  a  path  ( uw)Q  G  values(<7,7r(0),u)  such  that  w  G  C.  For  any  node 
u  G  J7,  let  Q-path(w)  be  the  lowest  ranked  path  of  values (cr,7r(0),-u)  that  goes  directly 
to  C.  Finally,  we  define  RQ-paths(u)  to  be  the  set  of  paths  values (cr,7r(0),-u)  —  {Q- 
path(u)}.  By  Lemma  V.6,  if  P  G  RQ-paths(w),  we  can  write  this  path  as  P  =  R 
Q-path(u)  where  R  is  a  path  of  the  form  (u  W\  W2--.v)  where  v  G  U,Wi  U  and 
Q-path(u)  is  a  path  that  leads  directly  to  some  fixed  node  w  G  C.  We  denote  the 
set  of  all  paths  of  the  form  (u  W\  w^-.-v)  as  R-paths(u).  Note  that  for  each  path 
of  RQ-paths(w),  there  is  a  corresponding  subpath  in  R-paths(u).  Finally,  for  such 
a  path  P ,  we  define  entering(P)  as  the  node  v  that  enters  C  by  routing  through  w. 
Some  of  this  terminology  is  presented  in  Figure  21. 
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Figure  21.  An  Illustration  of  Some  Terminology 

Definition:  Complete  Dispute  Wheel  Let  S  be  an  instance  of  SPP.  We 
define  a  dispute  wheel  of  to  be  complete  and  denote  it  0(5)  if  for  all  u  G  0(5),  we 
have  P“spp(G(S))=  'P's-  (f°r  each  node  in  the  dispute  wheel,  all  available  paths  for  the 
instance  5  are  contained  inside  the  dispute  wheel) 
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3.  Existence  of  Dispute  Wheel  for  an  Unsafe  Instance 
of  SPP 

The  following  lemmas  will  be  needed  to  show  that  every  unsafe  instance  of 
SPP  contains  a  dispute  wheel. 

Lemma  V.2.  If  every  node  of  a  digraph  G  has  an  outgoing  degree  of  exactly 
one,  the  graph  must  contain  a  cycle. 

Proof.  We  will  show  that  this  is  true  by  induction  on  the  number  of  nodes  of 
the  graph,  which  will  be  denoted  by  i.  There  are  also  exactly  i  edges.  Let  W{i)  be 
the  induction  hypothesis  that  every  graph  with  i  nodes  contains  a  cycle  if  the  graph 
has  i  edges. 

Base  Case.  Let  i  =  1.  This  one  element  graph  contains  a  cycle,  because  any 
outgoing  edge  from  the  single  node  must  be  to  itself.  Therefore  W(  1)  is  true. 

Induction  Step.  Suppose  every  graph  with  i  nodes  and  edges  contains  a  cycle. 

Let  G  be  some  arbitrary  graph  with  i  +  1  nodes  and  i  +  1  edges.  Let  u  be  some 
arbitrary  edge.  If  u  has  an  outgoing  edge  to  itself,  then  G  must  contain  a  cycle  and 
W(i  +  1)  is  true.  Otherwise  suppose,  u  has  an  outgoing  edge  to  some  node  v. 

In  our  first  case,  suppose  u  has  no  incoming  edges.  If  we  remove  u  and  the 
outgoing  edge  ( u  v ),  we  will  be  left  with  a  graph  with  exactly  i  nodes  and  edges.  This 
graph  must  have  a  cycle,  so  G  must  have  a  cycle  and  W(i  +  1)  must  be  true. 

In  our  second  case,  suppose  u  has  one  or  more  incoming  edges  from  nodes 
x,  y,  z....  For  each  such  edge  {x  u),  (y  u),  (z  u)...,  we  replace  the  edge  with  (x  v),  (y  v),(z  v).... 
Finally,  we  remove  node  u  and  the  edge  (u  v).  We  are  left  with  a  graph  with  i  nodes 
and  i  edges.  This  graph  must  contain  a  cycle.  This  implies  that  G  must  have  also 
contained  a  cycle.  We  have  examined  all  cases  and  W(i)  =>■  W{i  +  1). 

By  the  principle  of  induction,  if  every  node  of  a  digraph  G  has  an  outgoing 
degree  of  exactly  one,  the  graph  must  contain  a  cycle. 

□ 
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Now,  suppose  instead  of  having  an  outgoing  degree  of  exactly  one,  a  graph  has 
an  outgoing  degree  of  one  or  more.  The  graph  must  still  contain  a  cycle,  because  we 
are  introducing  additional  edges. 

A  strongly  connected  component  is  a  maximal  subgraph  of  a  digraph  such  that 
each  element  of  the  subgraph  can  reach  every  other  element  of  the  subgraph. 

Lemma  V.3.  Let  G  =  (1/,  E)  be  a  digraph.  G  has  at  least  one  strongly 
connected  component  with  no  outgoing  edges  to  other  strongly  connected  components. 

Proof.  We  use  proof  by  contradiction.  Suppose  every  strongly  connected  com¬ 
ponent  had  at  least  one  outgoing  edge  to  another  strongly  connected  component  and 
there  are  n  strongly  connected  components.  Every  digraph  may  be  decomposed  com¬ 
pletely  into  strongly  connected  components,  creating  another  digraph  of  strongly 
connected  components  [Ref.  3]  .  If  every  connected  component  has  at  least  one 
outgoing  edge  to  another  strongly  connected  component,  there  must  be  a  cycle  of 
strongly  connected  components  by  Lemma  V.2.  However,  this  reaches  a  contradic¬ 
tion  because  the  cycle  of  connected  components  would  itself  be  a  larger  connected 
component.  Therefore,  There  must  be  at  least  one  strongly  connected  component 
with  no  outgoing  edges  to  other  strongly  connected  components. 

□ 

For  any  unsafe  instance  of  SPP,  we  show  how  a  dispute  wheel  n  may  be 
created.  It  is  possible  that  more  than  one  dispute  wheel  may  be  generated  by  the 
procedure.  A  similar  proof  was  given  from  [Ref.  12]  as  Theorem  V.9. 

A  closed  walk  is  a  path  on  a  component  or  graph  such  that  the  path  visits 
every  node  and  edge  at  least  once,  and  begins  and  ends  with  the  same  node. 

Lemma  V.4.  Let  S  be  an  unsafe  instance  of  SPP.  S  has  a  dispute  wheel. 

Proof.  Let  V  be  the  set  of  paths  which  contains  the  every  path  P  G  {R- 
paths(ii)  |ii  6  U}.  Let  G(V)  be  the  graph  induced  by  taking  all  the  edges  and  nodes 
of  the  paths  of  V.  Each  node  u  has  one  or  more  paths  in  R -paths (it)  and  must 
have  an  outgoing  degree  of  one  or  greater  in  G{ V).  Furthermore,  consider  any  node 
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v  along  a  path  R-paths(-u)  such  that  v  G  O  —  U.  This  node  must  also  have  an 
outgoing  degree  of  at  least  one  in  G( V),  because  it  is  along  a  path,  and  it  can’t 
be  the  last  node  of  the  path,  because  that  node  is  in  U.  By  Lemma  V.3,  G(V) 
must  contain  at  least  one  strongly  connected  component  that  contains  no  outgoing 
edges  to  other  strongly  connected  components.  Furthermore,  this  strongly  connected 
component  must  contain  at  least  one  node  u  G  U.  This  is  because  if  it  contains  a 
node  dG  O  —  U,  this  node  will  have  a  path  to  a  node  in  u,  which  must  belong  to  the 
same  strongly  connected  component.  Let  C  be  such  a  strongly  connected  component 

We  claim  that  we  can  generate  a  dispute  wheel  from  C.  If  we  conduct  a  closed 
walk  on  the  resulting  graph,  we  will  have  visited  each  node  u  G  C  fl  U  and  each 
path  P  G  (R-paths('u)l'U  G  C  fl  U}  at  least  once.  We  form  our  dispute  wheel  by 
beginning  the  walk  with  an  arbitrary  node  u  G  C  fl  U  which  we  take  to  be  uq.  We 
take  Q0  =  Q-path(u0)  and  R0  to  be  the  path  of  R-paths(u)  we  take  first.  For  each 
subsequent  node  v  G  C  fl  U,  if  this  is  the  ith  time  we  have  reached  a  node  in  C  DU, 
we  take  tq  =  v,  Q0  =  Q-path(nfc),  and  finally,  we  take  the  next  path  traveled  to  be 
R,l.  This  process  terminates  at  the  end  of  our  closed  walk.  We  take  the  last  node 
reached  u  to  be  Uk  =  uq. 

We  must  now  show  that  we  have  generated  a  dispute  wheel.  Clearly,  properties 
1-3  of  a  dispute  wheel  have  been  satisfied,  otherwise  those  paths  would  not  occur 
infinitely  often.  Finally,  for  any  paths  RiQi+ 1  and  Qi,  \Ui{Qi)  <  \Ui {RiQi+\) ,  because 
otherwise  node  iq  would  never  switch  paths  away  from  Qi  because  that  path  is  always 
available. 

□ 

4.  All  Dispute  Wheels  Robust  and  Complete  Implies 
Safety 

In  the  following  lemma,  we  show  that  there  is  a  time  where  all  paths  that  do 
not  occur  infinitely  often,  can  no  longer  be  path  assignments  for  any  node. 
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Lemma  V.5  (Flushing  Paths  That  Do  Not  Occur  Infinitely).  Let  S  be  an 
unsafe  instance  of  the  stable  paths  problem.  Let  w  be  a  node  in  V.  Suppose  that 
P  qL  values(cr,7r(0),zc).  Then  there  is  a  time  tf  after  which  no  finite  path  of  the  form 
QP  belongs  in  n (t). 

Proof.  We  use  proof  by  induction  on  the  length  of  Q.  Let  Z(i)  be  the  predicate 
that  there  exists  a  time  tt  after  which  no  path  of  the  form  QP  belongs  in  7r (tf)  such 
that  Q  has  length  i  . 

Base  Case.  Let  i  —  0.  After  time  tci  the  node  w  can  only  update  its  assignment 
to  a  path  in  values(cr,7r(0),iy).  Node  w  is  activated  infinitely  often.  Let  t0  =  tw  >  tc 
be  the  next  time  node  w  is  activated  which  is  after  the  time  tc.  After  time  tw  =  t,0 
there  can  be  no  path  of  the  form  QP  such  that  Q  has  length  0.  Z{ 0)  is  true. 

Induction  Step.  Suppose  Z(i)  is  true.  There  exists  a  time  tt  after  which  no 
path  of  the  form  QP  belongs  in  7 r(L)  such  that  Q  has  length  i.  Let  v  be  a  node  such 
that  7r (£$,  v)  =  QP  where  Q  has  length  i  +  1.  Node  v  is  activated  infinitely  often.  Let 
tv  >  ti  be  the  next  time  node  v  is  activated.  After  time  tv  there  can  be  no  path  of  the 
form  QP  because  v  can  no  longer  adopt  this  path.  For  all  v  such  that  7 r(L,  v)  =  QP 
where  Q  has  length  i  +  1,  let  tl+\  be  max(tv).  After  time  tl+\  there  can  be  no  path 
of  the  form  QP  such  that  Q  has  length  i  +  1.  Z(i)  =>-  Z(i  +  1). 

Our  predicate  Z(i)  is  true  for  all  i  >  0.  By  the  principle  of  induction,  we  have 
shown  that  there  exists  a  time  t  after  which  no  finite  path  of  the  form  QP  belongs  in 
n(t). 

□ 

In  this  theorem,  we  show  that  for  any  path  that  occurs  infinitely  often,  all 
subpaths  must  also  occur  infinitely  often. 

Lemma  V.6.  For  some  node  u,  if  P  G  values(cr,7r(0),w)  where  P  =  (w0Wi...Wk) 
,  then  for  all  Wi,  P[u>iO]  G  values(a,7r(0),zcj). 

Proof.  If  P[iUiO}  (f  values (<r,7r(0),zci)  and  P  G  values (cr,7r(0),tt)  there  would  be 
a  contradiction,  because  by  V.5  the  path  P  should  have  been  flushed  after  some  time 
t. 
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□ 

In  the  following  theorem  we  give  our  main  result  of  this  subsection.  We  show 
that  for  an  instance  of  SPP,  if  all  dispute  wheels  are  robust  and  complete,  then  the 
instance  of  SPP  is  safe. 

Lemma  V.7.  Let  S  be  an  instance  of  the  stable  paths  problem.  If  every  dispute 
wheel  of  S  is  complete  and  robust,  then  S  is  safe. 

Proof.  We  will  use  proof  by  contradiction.  Suppose  every  dispute  wheel  of  S 
is  complete  and  robust,  but  S  is  not  safe. 

By  Lemma  V.4,  we  know  that  S  must  contain  a  dispute  wheel  that  has  nodes 
which  oscillate  infinitely  often.  Furthermore,  we  have  assumed  that  this  dispute 
wheel,  ©(S'),  is  complete.  We  will  show  that  ©(S')  can  not  be  robust,  which  will  be 
our  contradiction. 

Let  the  path  assignment  7r(0)  and  the  activation  sequence  cr  be  unsafe  for 
S.  We  will  use  induction  to  show  that  we  can  find  an  initial  path  assignment  and 
activation  sequence  such  that  SPPQ(S)  is  not  safe.  As  usual,  let  n{i)  define  the  path 
assignment  at  time  i  for  S  under  the  activation  sequence  cr.  Let  tf  be  the  time  where 
all  paths  have  been  flushed  out  of  the  system  as  in  V.5.  Let  ^f(z)  define  the  path 
assignment  for  SPP(0(S))  with  the  activation  sequence  a  for  all  times  i  >  tf. 

At  time  t  =  tf,  we  let  %(Tf)  have  the  following  path  assignments.  For  all 
u  G  ©(*5),  let  ^(u,tf)  =  7T (u,tf)  .  For  all  w  qL  ©(S'),  these  nodes  do  not  occur  in 
SPP  (0(5)). 

Let  T(i)  be  the  predicate  that  at  time  i  the  following  holds  true.  For  all 
u  G  0(S),  ^f(u,  i)  =  7 . 

Base  Case.  At  time  tf  we  let  %(Tf)  have  the  above  path  assignments,  so  we 
know  T(z)  is  true. 

Induction  Step.  Suppose  T(z)  is  true.  We  know  that  for  all  u  G  0(S),  ^f(zz,  z)  = 
7i(u,i).  Linder  the  activation  sequence  a,  at  time  i  +  1  the  nodes  Ui+ 1  are  activated. 
Each  node  in  Ul+\  will  be  denoted  by  zzfc)i+1. 
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Suppose  Ukti+ 1  €  ©(S').  This  activation  will  cause  node  w^i+i  to  take  the 
path  Tr(ukti+i,i  +  1)  =  best(choices(7r(?,),-ufcii+1,f)).  For  node  u^i+i,  we  know  that 
VUk’i+1e(s)  —  VUk’i+1  5,  because  O(S)  is  a  complete  dispute  wheel.  Because  the  next- 
hop  of  every  availible  path  is  in  0(S)  and  by  our  induction  hypothesis,  we  must  then 
have  choices(^f(i),rtfe)j+i,i)  =  choices(7r(z),Mfc)j+i,f)  .  Therefore,  best(choices(^[(*),Mfcjj+i,i)) 
best(choices(7r(i),wfcji+i,*)),  and  1(m  i+i,  i  +  1)  =  7r(ukti+i,  i  +  1). 

Suppose  Uk,i+ 1  fL  ©(S).  This  node  does  not  occur  in  SPP(0(S)).  Therefore, 

T(i)  =>  Y(i  +  1). 

We  have  used  induction  to  show  that  the  nodes  of  SPP(0(S))  will  have  the 
same  sequence  of  path  assignments  as  it.  We  can  use  n(tf)  and  the  subsequence  of  a 
beginning  with  the  PJ1  element  with  all  elements  uk,i+ i  0(S)  removed  as  an  initial 
path  assignment  and  activation  sequence  for  SPP(©(S))  that  will  cause  some  nodes 
of  ©(S')  to  oscillate  indefinitely.  SPP(0(S))  is  not  safe. 

However,  we  have  reached  a  contradiction,  because  we  assumed  every  dispute 
wheels  was  robust,  and  thus  can’t  be  unsafe.  If  every  dispute  wheels  of  S  is  complete 
and  robust,  then  S  must  be  safe. 

□ 

E.  A  WEAKER  SUFFICIENT  CONDITION  FOR  SPP 
ROBUSTNESS 

The  following  lemma  is  important  for  our  main  theorem. 

Lemma  V. 8.  Let  S  be  mi  instance  of  the  stable  paths  problem.  If  every  dispute 
wheel  of  S  is  robust  and  complete,  then  every  dispute  wheel  of  all  subinstances  of  S 
is  robust  and  complete. 

Proof.  Let  S  =  S  =  ( G,V,A )  be  an  an  instance  of  the  stable  paths  problem 
where  G  =  (V,E).  Let  E'  C  E  and  SPP(£")  be  a  subinstance  of  the  stable  paths 
problem.  Let  n  be  any  dispute  wheel  of  SPP (Ef).  Any  dispute  wheel  n,  must  also 
be  a  dispute  wheel  for  S,  which  can  be  denoted  by  ©(S').  Because  of  our  assump¬ 
tion,  SPP(0(S))  is  robust.  Therefore,  SPP(n)  must  also  be  robust,  because  it  is  a 
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subinstance  of  SPP(0(S')).  Therefore  II  is  a  robust  dispute  wheel.  Finally,  because 
for  each  node  ufll,  Vue,  C  Vu  =  Vu  s  spp(0(S))  ,  n  must  also  be  complete  dispute 
wheel. 

□ 


Our  main  theorem  gives  our  new  sufficient  condition  for  robustness. 

Theorem  V.9  (A  Weaker  Sufficient  Condition  for  SPP  Robustness).  Let  S 
be  an  instance  of  SPP.  If  every  dispute  wheel  of  S  is  complete  and  robust,  then  S  is 
robust. 

Proof.  We  know  by  Lemma  V.8,  that  every  dispute  wheel  of  all  subinstances 
of  S  will  be  complete  and  robust.  Therefore,  we  know  by  Lemma  V.l  and  Lemma 
V.7,  that  S,  and  all  subinstances  of  S  will  be  uniquely  solvable  and  safe  respectively. 
Therefore  S  must  be  robust. 

□ 

We  believe  that  the  above  results  are  true  for  SPVP,  as  well.  We  believe  a 
similar  inductive  proof  could  be  conducted  for  Lemma  V.7  using  SPVP.  In  such  a 
proof,  the  sequence  of  path  assignments  for  nodes  of  a  complete  dispute  wheel  would 
be  the  same  as  for  the  same  nodes  in  the  total  instance  of  SPP. 

We  compare  our  condition  for  robustness  to  the  existing  sufficient  condition 
for  robustness,  which  is  having  no  dispute  wheel.  If  an  instance  of  SPP  has  no  dispute 
wheel,  then  it  satisfies  our  condition.  However,  an  instance  of  SPP  may  satisfy  our 
condtion,  but  not  the  condition  of  having  no  disptute  wheels.  Therefore,  our  condition 
is  weaker  than  the  condition  of  having  no  dispute  wheel.  Unfortunately,  our  weaker 
sufficient  condition  is  not  a  necessary  and  sufficient  condition  for  robustness  because 
there  exist  instances  of  SPP  that  our  robust,  but  do  not  meet  our  condition. 

Consider  the  instance  of  SPP  given  in  Figure  22  which  we  will  call  “COUN- 
TEREX”.  This  instance  of  SPP  contains  the  dispute  wheel  depicted  in  Figure  19 
which  we  will  call  Hcounterex-  However,  this  dispute  wheel  is  not  complete.  For 
COUNTEREX,  the  path  (1  4  0)  is  available  at  node  1.  However,  for  the  derived 
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instance  SPP(nCouNTEREx)>  the  path  (14  0)  can  not  be  available  at  node  1  because 
node  4  does  not  belong  to  the  dispute  wheel.  COUNTEREX  does  not  meet  our 
condition,  but  we  claim  COUNTEREX  robust.  Just  because  an  instance  of  SPP  is 
robust,  this  does  not  necessarily  mean  that  it  meets  our  condition.  Therefore,  our 
condition  can  not  be  necessary  and  sufficient. 


Figure  22.  COUNTEREX:  A  Robust  Instance  of  SPP  that  Does  Not  Meet  Our 
Condition 

Figure  23  compares  our  condition  with  the  condition  in  previous  work. 


All  Robust  Instances  of  SPP 


All  Dispute  Wheels  are  Complete 
and  Robust 

(presented  in  this  paper) 

No  Dispute  Wheels 
(Griffin  and  Wilfong) 

Figure  23.  Conditions  to  Guarantee  SPP  Robustness 
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F.  APPLICATION  OF  MAIN  THEOREM 

We  have  shown  that  for  any  instance  of  SPP,  if  every  dispute  wheel  is  complete 
and  robust,  then  the  instance  is  robust.  In  order  to  apply  this  theorem,  two  steps 
must  be  taken.  First,  given  an  instance  of  SPP,  we  must  find  all  dispute  wheels. 
Second,  once  dispute  wheels  have  been  found,  we  must  show  that  each  dispute  wheel 
is  robust  and  complete. 

1.  Finding  Dispute  Wheels 

In  order  to  apply  our  main  theorem,  we  must  find  all  dispute  wheels  for  a  given 
instance  of  SPP.  In  Section  I,  we  introduced  Class-Based  Path  Vector  systems  as  an 
abstraction  of  BGP  and  SPP  that  meet  well-characterized  contraints  based  upon 
the  relationships  between  nodes.  Ramachandran  and  Jaggard  gave  a  centralized 
polynomial  time  algorithm  (Algorithm  4.1  [Ref.  18])  that  determines  all  directed 
cycles  of  troublesome  classes  which  correspond  to  potential  dispute  wheels.  They 
proved  that  their  algorithm  was  complete.  They  left  open  the  problem  of  determining 
exactly  which  dispute  wheels,  if  any,  occur  for  a  directed  cycle.  We  could  take  the 
troublesome  cycle,  and  make  sure  that  it  meets  some  set  of  constraints  such  that  if 
the  cycle  does  create  a  dispute  wheel,  that  dispute  wheel  is  robust  and  complete.  An 
example  of  such  conditions  are  given  in  the  next  session. 

Unfortunately,  this  approach  only  works  for  instances  of  the  stable  paths  prob¬ 
lem  which  meet  the  constraints  of  class-based  path- vector  systems. 

2.  Constraints  that  Guarantee  Robustness  Despite  the 
Presence  of  a  Dispute  Wheel 

To  guarantee  robustness  for  an  instance  of  SPP,  all  dispute  wheels  must  be 
robust  and  complete.  The  computational  complexity  of  determining  robustness  of 
general  instances  of  SPP  remains  an  open  problem  [Ref.  12],  It  may  be  NP-Hard. 
Therefore,  we  would  like  to  develop  global  and  local  contraints  that  guarantee  robust¬ 
ness,  despite  the  presence  of  a  dispute  wheel.  If  all  dispute  wheels  for  an  instance  of 
SPP  followed  these  contraints,  then  the  instance  would  be  guaranteed  to  be  robust. 


63 


We  introduce  a  set  of  contraints  (“Set  A”)  on  SPP  that  is  guaranteed  to  be 
robust  and  have  a  dispute  wheel. 

“Set  A”  of  Contraints  on  S  =  (G,  V  ,  A) 

1.  V  —  {d,  0, 1,  ...n  —  1}  where  d  is  the  origin  and  n  >  3 

2.  E  =  {(1  d),  (2  d),  ...(n  —  1  d),  (n  d ),  (1  2),  (2  3), ...,  (n  —  2  n  —  1),  (n  1)} 

3.  For  each  node  k  e  V  —  {d},  Vk  =  {( k  d ),  (/c  k  +  1  d),  (k  k  +  1  k  +  2  d),  ...(A; 
k  +  1  k  +  2...k  —  1  d)} 

4.  (A;  d)  is  the  highest  ranked  path  at  every  node  k 

5.  For  all  other  paths,  \k{P\)  >  \k{P2)  if  Pi  is  longer  than  P2. 

6.  For  each  node  k,  k  —  k  +  n, 

If  an  instance  of  SPP  meets  these  contraints,  it  will  contain  the  dispute  wheel 
of  size  72  —  1  where  Ui  —  i  +  1,  Qi  =  (i  i  +  1  0),  and  Rt  —  (i  i  +  1).  The  purpose  of 
“Set  A”  is  to  illustrate  that  there  does  exist  some  sets  of  general  contraints,  which 
guarantee  robustness  depsite  the  presence  of  a  dispute  wheel. 

Theorem  V.10.  If  an  instance  of  SPP  meets  “Set  A  ”  of  constraints,  then  it 
is  robust. 

Proof.  We  must  show  that  the  instance  of  SPP  is  uniquely  solvable  and  safe 
under  any  combination  of  edge  removals.  If  any  edge  (k  k  +  1)  is  removed,  there  is  no 
possible  way  the  subinstance  still  contains  a  dispute  wheel,  so  the  subinstance  is  safe 
and  uniquely  solvable.  If  all  72  —  1  edges  of  the  form  (d  k)  are  removed,  then  there 
is  a  unique,  safe  solution  where  every  node  k  gets  the  empty  path  assignment.  We 
now  consider  the  cases  where  between  1  and  n  —  2  edges  are  removed,  but  all  edges 
of  the  form  (k  k  +  1)  are  present.  Without  loss  of  generality,  we  assume  the  edge  (1 
d)  is  present.  We  use  induction  on  the  edges  to  prove  that  every  node  has  a  unique 
solution  and  is  guaranteed  to  converge  to  it  after  some  finite  number  of  activations. 
Our  induction  hypothesis  Z(i)  is  that  node  i  has  a  unique  solution  and  is  guaranteed 
to  converge  to  its  unique  solution  after  some  Unite  number  of  activations. 
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Base  Case.  Node  1  has  the  unique  solution  (1  d)  because  it  is  the  highest 
ranked  path  and  is  also  always  available.  Also,  node  1  is  activated  infinitely  often,  so 
it  will  converge  after  some  finite  number  of  activations. 

Inductive  Step.  We  assume  that  node  i  has  a  unique  solution  and  will  converge 
to  it  after  some  finite  number  of  activations.  We  would  like  to  show  that  node  i— 1  also 
has  a  unique  solution  and  will  converge  to  it  after  some  finite  number  of  activations. 
Suppose  the  edge  (i  —  1  d)  has  not  been  removed.  This  case  is  the  same  as  the  base 
case,  therefore  Z(i  —  1)  is  true.  Suppose  the  edge  (i  —  Id)  has  been  removed.  After 
some  finite  number  of  activations,  node  i  will  converge  to  some  path  Pl  —  (i  [i  +  1] 
[z  +  2]  ...d).  This  path  can  not  be  (i  i  +  —  1  d)  because  edge  (i  —  1  d)  is  unavailable. 

Therefore,  the  path  (i  —  1  i)P\  is  available  at  node  i  —  1  because  of  the  “Set  A”  of 
constraints.  Because  this  is  the  only  available  path,  and  the  path  assignment  of  node 
i  is  unique,  the  path  assignment  of  node  i  —  1  must  also  be  unique.  Furthermore, 
because  each  node  is  activated  infinitely  often,  node  i  —  1  will  be  activated  sometime 
after  node  i  recieves  its  path  assignment,  so  node  i  —  1  will  converge  after  some  finite 
number  of  activations  as  well.  Z(i)  =>  Z(i  —  1). 

By  the  principle  of  induction,  all  nodes  in  the  subinstance  have  a  unique  path 
assignment  and  are  guaranteed  to  converge  to  it  after  some  finite  number  of  activations 
when  between  1  and  n  —  2  edges  of  the  form  (i  d)  fail.  Therefore,  if  an  instance  of 
SPP  meets  “Set  A”  of  contraints,  it  is  robust  under  all  cases  of  edge  failures. 

□ 

We  compare  how  “Set  A”  compares  with  existing  robust  operational  guidelines. 
Because  all  existing  guidelines  are  based  upon  an  instance  of  SPP  having  no  dispute 
wheels,  “Set  A”  is  disjoint  from  existing  guidelines  as  depicted  by  Figure  24.  Note 
that  the  set  of  “All  Robust  Operational  Guidelines”  does  not  actually  exist,  because 
no  necessary  and  sufficient  condition  for  robustness  has  been  found. 

Clearly,  the  conditions  of  “Set  A”  create  a  complete  dispute  wheel.  Any  dis¬ 
pute  wheel  generated  by  these  conditions  will  contain  every  node  and  the  availible 
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paths  at  every  node  are  contained  in  the  dispute  wheel 


All  Robust  Operational  Guidelines 


;  “Set  A”  I 

1  (New  Operational  Guidelines  that  J 

_  cp,n  tain_dj  sjou  te_w  hee  Q  .....  J 

Griffin  and  Wilfong 

Operational  Guidelines  with  no  Dispute  Wheel 

Jaggard  and  Ramachandran 
(Class-Based, 

exact  condition  for  no  dispute  wheel) 


Gao  and  Rexford 
(Customer-Provider, 
peer-to-peer) 


Figure  24.  Robust  Operational  Guidelines  for  SPP  and  BGP 


Unfortunately,  the  constraints  given  by  “Set  A,”  are  too  strict.  There  exist 
other  instances  of  SPP  that  contain  robust  dispute  wheels.  We  would  like  to  in¬ 
vestigate  the  most  general  constraints  possible  that  guarantee  the  robustness  and 
completeness  of  dispute  wheels. 

In  general,  our  results  give  could  be  applied  to  give  BGP  operators  more  flex¬ 
ibility.  Operators  could  follow  existing  operational  guidelines,  or  they  could  follow 
new  operational  guidelines  that  are  guaranteed  to  produce  robust  and  complete  dis¬ 
pute  wheels.  By  following  either  such  guidelines,  the  system  of  BGP  routers  will  be 
provably  robust. 


VI. 


CONCLUSION  AND  FUTURE  WORK 


In  this  paper,  we  have  extended  previous  work  on  interdomain  routing  by  fo¬ 
cusing  on  the  stable  paths  problem.  In  particular,  we  introduce  a  new  sufficient  con¬ 
dition  for  interdomain  routing  that  guarantees  robustness.  This  condition  is  weaker 
than  those  previously  published.  We  also  compare  various  models  of  BGP  behavior. 
We  show  that  such  models  do  not  necessarily  have  equivalent  definitions  of  safety. 
We  also  show  that  such  models  do  not  necessarily  match  each  other  in  terms  of  the 
possible  path  assignments  each  model  may  reach  for  the  same  instance  of  the  stable 
paths  problem. 

There  are  still  a  large  number  of  open  problems  pertaining  to  interdomain 
routing  and  robustness.  The  condition  for  robustness  we  have  introduced  is  not  likely 
to  be  the  most  general  condition  for  robustness.  Ramachandran  conjectured  that  no 
general  set  of  conditions  can  capture  all  robust  instances  of  the  stable  paths  problem 
(Conjecture  4.5.3  [Ref.  21]).  Either  a  necessary  and  sufficient  condition  for  the  stable 
paths  problem  will  have  to  be  found,  or  this  conjecture  will  need  to  be  proven. 

As  mentioned  in  Chapter  V,  we  believe  our  main  results  could  also  be  proven 
using  the  simple  path  vector  protocol.  A  formal  proof  of  this  would  give  greater 
confidence  that  our  results  can  undoubtedly  be  applied  to  BGP. 

During  the  research  for  this  thesis,  we  were  unable  to  prove  whether  or  not 
safe  (MNASM)  implies  safe(SPVP).  Either  a  counterexample  will  need  to  be  found, 
or  some  proof  will  need  to  be  made. 

The  problem  of  devising  more  general  conditions  than  those  given  in  “Set 
A”  remains  open.  These  conditions  are  strict,  and  it  is  possible  that  much  broader 
conditions  based  upon  our  main  result  could  be  given.  Once  broad  conditions  have 
been  constructed,  it  would  be  useful  to  convert  such  conditions  to  guidelines  for  BGP 
operators. 
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APPENDIX.  AN  EXAMPLE  OF  A  ROUTER 

CONFIGURATION 


Current  configuration  :  1289  bytes 
! 

version  12.1 

service  timestamps  debug  uptime 
service  timestamps  log  uptime 
no  service  password-encryption 
! 

hostname  Bart 


memory-size  iomem  15 
ip  subnet-zero 


interface  EthernetO/O 
ip  address  10.0.6.1  255.255.255.0 

! 

interface  Ethernetl/0 
ip  address  10.0.4.1  255.255.255.0 

! 

interface  Ethernet 1/1 
ip  address  10.0.1.2  255.255.255.0 

! 

interface  Ethernetl/2 
ip  address  10.64.10.1  255.255.255.0 
shutdown 

! 

interface  Ethernetl/3 
ip  address  10.2.1.17  255.255.255.248 
shutdown 
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router  bgp  101 
no  synchronization 
bgp  log-neighbor-changes 
timers  bgp  5  15 
redistribute  connected 
neighbor  10.0.1.1  remote-as  102 
neighbor  10.0.1.1  route-map  MARGE  in 
neighbor  10.0.1.1  filter-list  103  in 
neighbor  10.0.4.2  remote-as  103 
neighbor  10.0.6.2  remote-as  100 
neighbor  10.0.6.2  route-map  HOMER  in 
no  auto- summary 

i 

ip  classless 

ip  route  100.0.0.0  255.255.255.255  10.0.1.1 
no  ip  http  server 

ip  as-path  access-list  100  permit  ~100\$ 
ip  as-path  access-list  102  permit  “10 
ip  as-path  access-list  103  deny  103 
ip  as-path  access-list  103  permit  . * 

i 

access-list  1  permit  10.0.1.1 
route-map  MARGE  permit  10 
match  as-path  102 
set  local-preference  200 

i 

route-map  HOMER  permit  10 
match  as-path  100 
set  local-preference  100 


line  con  0 
line  aux  0 
line  vty  0  4 
login 

i 

end 
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